Imaging apparatus, imaging method, and program

ABSTRACT

An image processing apparatus including an interface that receives an input identifying a subject, and a processor that generates a panoramic image based on captured image data so that the subject is positioned in a reference position in the panoramic image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and is based upon and claims thebenefit of priority under 35 U.S.C. § 120 for U.S. Ser. No. 13/276,546,filed Oct. 19, 2011 and claims the benefit of priority under 35 U.S.C. §119 to Japanese Patent Application JP 2010-252612 filed in the JapanPatent Office on Nov. 11, 2010, the entire content of which is herebyincorporated by reference.

BACKGROUND

The present disclosure relates to an imaging apparatus, and specificallyrelates to an imaging apparatus and an imaging method which generate apanoramic image, and a program causing a computer to execute thismethod.

In recent years, there have come into widespread use imaging apparatusessuch as digital still cameras which image a subject such as scenery orthe like to generate images (image data), and record the generatedimages as image files. Also, there has been proposed an imagingapparatus which consecutively generates multiple images in chronologicalorder, and generates a panoramic image including a subject with arelatively wide range using these generated multiple images.

For example, there has been proposed an imaging apparatus whereinmultiple images are generated while moving the imaging apparatus withthe backward of the imaging apparatus (e.g., the position of aphotographer) as an axis so as to draw an arc, and a panoramic image isgenerated using these multiple images (e.g., see Japanese UnexaminedPatent Application Publication No. 2009-268037).

SUMMARY

According to the above-mentioned related art, in a state in which a userholds an imaging apparatus in the hand, the imaging operation of apanoramic image may be performed by moving the imaging apparatus so asto draw an arc, and accordingly, the user may perform imaging operationthereof in a relatively easy manner.

In this way, in the event of performing the imaging operation of apanoramic image using the above-mentioned related art, the compositionof the panoramic image is determined according to a start instruction ofimaging operation by the user's operations, for example. Accordingly, inthe event of performing the imaging operation of a panoramic image,after overlooking the circumference, the user has to confirm a subjectserving as an imaging object for a panoramic image, and to recognize thestart position and end position of imaging operation, for example.Subsequently, after performing an imaging operation start instruction atthe start position of imaging operation, the user performs the imagingoperation of a panoramic image by moving the imaging apparatus to theend position of imaging operation.

However, in the event of performing the imaging operation of a panoramicimage based on a user's confirmation in this way, the composition may bedifferent from that which the user has intended. For example, there maybe a case where an object which the user intends to situate in thecenter position in a panoramic image is not situatable in the centerposition in a panoramic image. In this case, for example, the userperforms multiple times of imaging operation through a trial and errorprocess, or compromises with a panoramic image having an unintendedcomposition.

It has been found to be desirable to enable readily generating a user'sfavorite panoramic image.

According to a first exemplary embodiment, the disclosure is directed toan image processing apparatus including an interface that receives aninput identifying a subject, and a processor that generates a panoramicimage based on captured image data so that the subject is positioned ina reference position in the panoramic image.

According to another exemplary embodiment, the disclosure is directed toan image processing apparatus that includes an interface that receivesan input identifying a subject, an imaging unit that captures image datafor generating a panoramic image, and a processor that generates thepanoramic image based on the captured image data so that the subject ispositioned in a reference position in the panoramic image.

According to another exemplary embodiment, the disclosure is directed toan image processing method performed by an image processing apparatus.The method includes receiving, at an interface of the image processingapparatus, an input identifying a subject, and generating, at aprocessor of the image processing apparatus, a panoramic image based oncaptured image data so that the subject is positioned in a referenceposition in the panoramic image.

According to the above configurations, an excellent advantage may beprovided in that a user's favorite panoramic image can be readilygenerated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an internal configuration exampleof an imaging apparatus according to a first embodiment of the presentdisclosure;

FIG. 2 is a block diagram illustrating a functional configurationexample of the imaging apparatus according to the first embodiment ofthe present disclosure;

FIGS. 3A and 3B are perspective views illustrating the appearance of theimaging apparatus according to the first embodiment of the presentdisclosure;

FIG. 4 is a diagram illustrating a display screen example to bedisplayed on an input/output panel according to the first embodiment ofthe present disclosure;

FIGS. 5A and 5B are diagrams illustrating a setting screen example to bedisplayed on the input/output panel according to the first embodiment ofthe present disclosure;

FIGS. 6A and 6B are diagrams illustrating relationship between imagingoperation in the event of generating a panoramic image using the imagingapparatus according to the first embodiment of the present disclosure,and a panoramic image generated by this imaging operation;

FIGS. 7A and 7B are diagrams illustrating a setting screen example to bedisplayed on the input/output panel according to the first embodiment ofthe present disclosure;

FIGS. 8A and 8B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 9A and 9B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 10A and 10B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 11A and 11B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 12A and 12B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 13A and 13B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 14A and 14B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIG. 15 is a diagram illustrating an example of a swing operationassistance screen to be displayed on the input/output panel according tothe first embodiment of the present disclosure;

FIGS. 16A and 16B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 17A and 17B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 18A and 18B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 19A and 19B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 20A and 20B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 21A and 21B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 22A and 22B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 23A and 23B are diagrams schematically illustrating a transitionexample of the orientation of the imaging apparatus according to thefirst embodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIG. 24 is a flowchart illustrating an example of imaging controlprocessing according to the imaging apparatus according to the firstembodiment of the present disclosure;

FIG. 25 is a flowchart illustrating an example of imaging controlprocessing according to the imaging apparatus according to the firstembodiment of the present disclosure;

FIG. 26 is a flowchart illustrating an example of imaging controlprocessing according to the imaging apparatus according to the firstembodiment of the present disclosure;

FIGS. 27A and 27B are diagrams illustrating a display screen example tobe displayed on the input/output panel according to a second embodimentof the present disclosure;

FIGS. 28A and 28B are diagrams schematically illustrating an example ofthe orientation of the imaging apparatus according to the secondembodiment of the present disclosure, a display screen to be displayedon the input/output panel;

FIGS. 29A, 29B and 29C are diagrams schematically illustrating arelation example between the orientation of the imaging apparatusaccording to the second embodiment of the present disclosure, a displayscreen to be displayed on the input/output panel, and a referencesubject in a panoramic image;

FIGS. 30A, 30B and 30C are diagrams schematically illustrating arelation example between the orientation of the imaging apparatusaccording to the second embodiment of the present disclosure, a displayscreen to be displayed on the input/output panel, and a referencesubject in a panoramic image;

FIGS. 31A, 31B and 31C are diagrams schematically illustrating arelation example between the orientation of the imaging apparatusaccording to the second embodiment of the present disclosure, a displayscreen to be displayed on the input/output panel, and a referencesubject in a panoramic image;

FIGS. 32A, 32B and 32C are diagrams schematically illustrating arelation example between the orientation of the imaging apparatusaccording to the second embodiment of the present disclosure, a displayscreen to be displayed on the input/output panel, and a referencesubject in a panoramic image;

FIGS. 33A and 33B are diagrams schematically illustrating an example ofthe orientation of the imaging apparatus according to the secondembodiment of the present disclosure, and a display screen to bedisplayed on the input/output panel;

FIGS. 34A and 34B are diagrams illustrating a display screen example tobe displayed on the input/output panel according to a second embodimentof the present disclosure;

FIG. 35 is a flowchart illustrating an example of imaging controlprocessing procedures according to the imaging apparatus according tothe second embodiment of the present disclosure;

FIGS. 36A and 36B are diagrams illustrating a display screen example tobe displayed on the input/output panel according to the secondembodiment of the present disclosure; and

FIGS. 37A and 37B are diagrams schematically illustrating imagingoperation at the time of generating a panoramic image using the imagingapparatuses according to the first embodiment and second embodiment ofthe present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

Modes for implementing the present disclosure (hereafter, referred to asembodiments) will be described below. Description will be made inaccordance with the following sequence.

1. First Embodiment (imaging control: an example wherein the imagingoperation of a panoramic image is controlled so as to situate a user'sdesired subject in a reference position in the panoramic image)

2. Second Embodiment (imaging control: an example wherein the imagingoperation of a panoramic image is controlled so as to obtain a user'sdesired composition by situating a marker representing a referencesubject on a composition assistance image)

3. Modifications

1. First Embodiment Configuration Example of Imaging Apparatus

FIG. 1 is a block diagram illustrating an internal configuration exampleof an imaging apparatus 100 according to the first embodiment of thepresent disclosure. The imaging apparatus 100 includes a lens unit 111,an imaging unit 112, an imaging control unit 113, a temporary storageunit 114, an orientation detecting unit 121, an image apparatusorientation control unit 122, an image processing unit 130, an outputcontrol unit 140, a display unit 141, and an external output unit 142.Also, the imaging apparatus 100 includes an operation reception unit151, an input control unit 152, a UI (user interface) control unit 160,a playback control unit 170, and a recording medium 180. Note thatexchange between the units making up the imaging apparatus 100 isperformed via a bus 101. Also, the imaging apparatus 100 may be realizedby a digital still camera capable of imaging a subject to generatemultiple images (image data), and performing various types of imagingprocessing regarding these multiple images, for example.

The lens unit 111 is configured of multiple lenses (zoom lens, focusinglens, etc.) for condensing light from a subject, and light input fromthe subject via these lenses and an iris is supplied to the imaging unit112.

The imaging unit 112 converts incident light from a subject to generatean image (image data) under control of the imaging control unit 113, andsupplies the generated image to the imaging control unit 113.Specifically, an optical image of the subject input via the lens unit111 is formed on the imaging face of the imaging device, and the imagingdevice performs imaging operation in this state to generate an image(image data). Note that a CCD (Charge Coupled Device) sensor, a CMOS(Complementary Metal Oxide Semiconductor) sensor, and so forth may beused as the imaging device, for example.

The imaging control unit 113 controls the imaging operation by theimaging unit 112, and supplies the image (image data) generated by theimaging unit 112 to the image processing unit 130 via the temporarystorage unit 114.

The temporary storage unit 114 is image memory for temporarily storingthe image (image data) generated by the imaging unit 112, or an image(image data) at the time of image processing by the image processingunit 130. The temporary storage unit 114 is configured of DRAM (DynamicRandom Access Memory), for example.

The orientation detecting unit 121 detects change in the orientation ofthe imaging apparatus 100 by detecting the acceleration, motion,inclination, and so forth of the imaging apparatus 100, and outputsorientation change information relating to detected orientation changeto the imaging apparatus orientation control unit 122. The orientationdetecting unit 121 may inexpensively be realized by taking advantage ofa gyro sensor for shaking correction which is a ready-made component.Alternatively, a dedicated gyro sensor may also be employed. The angularvelocity of the imaging apparatus 100 is detected by these gyro sensors,and change in the orientation of the imaging apparatus 100 is detected.Note that an arrangement may be made wherein the acceleration, motion,inclination, and so forth of the imaging apparatus 100 are detectedusing another sensor other than gyro sensors (e.g., accelerationsensor), and based on this detection result, the orientation of theimaging apparatus 100 and change thereof are detected. The imagingapparatus orientation control unit 122 controls the orientationdetecting unit 121, and outputs the orientation information output fromthe orientation detecting unit 121 to the image processing unit 130 andUI control unit 160 and so forth.

The image processing unit 130 performs image processing such as encodingand so forth as to the image (image data) generated by the imaging unit112, and stores the image (image data) subjected to image processing inthe recording medium 180 as an image file.

For example, the image processing unit 130 performs image processing fordisplay regarding the image (image data) generated by the imaging unit112, and outputs the image subjected to image processing to the outputcontrol unit 140 to display on the display unit 141. Also, in the eventthat a panoramic image imaging mode has been set, the image processingunit 130 synthesizes the multiple images (image data) generated by theimaging unit 112 to generate a panoramic image, and stores thisgenerated panoramic image in the recording medium 180 as an image file.Also, the image processing unit 130 performs image processing such asdecoding processing and so forth of the image supplied from the playbackcontrol unit 170 via the temporary storage unit 114 at the time ofplayback of an image file stored in the recording medium 180.

The output control unit 140 performs output control relating to thedisplay unit 141 and external output unit 142. For example, the outputcontrol unit 140 displays the image (e.g., through image) output fromthe image processing unit 130 on the display unit 141. Also, the outputcontrol unit 140 displays each image (e.g., menu screen) on the displayunit 141 under control of the UI control unit 160. Also, the outputcontrol unit 140 synthesizes each image under control of the UI controlunit 160 (e.g., operation assistance image) as to the image output fromthe image processing unit 130 (e.g., through image) to display on thedisplay unit 141. Also, the output control unit 140 displays an imagefile stored in the recording medium 180 on the display unit 141 undercontrol of the playback control unit 170.

The display unit 141 is a display unit for displaying various imagessupplied from the output control unit 140. The display unit 141sequentially displays the image generated by the imaging unit 112 (whatwe might call monitoring display), for example. Also, the display unit141 displays the image corresponding to an image file stored in therecording medium 180, for example. Note that a display panel such as anorganic EL (Electro Luminescence) panel or the like may be employed asthe display unit 141, for example. Alternatively, a touch panel whichthe user can input operations by contacting or approaching his/herfinger on the display surface may be employed, as illustrated in FIGS.3A and 3B.

The external output unit 142 outputs various types of informationsupplied from the output control unit 140 to an external device.

The operation reception unit 151 is an operation reception unit forreceiving operation input operated by the user, and outputs a signalaccording to the received operation input to the input control unit 152.For example, with the panoramic image imaging mode, operation memberssuch as a shutter button 210 (shown in FIGS. 3A and 3B) for instructingstart of imaging operation for generating a panoramic image, and soforth are provided to the imaging apparatus 100. Also, the operationreception unit 151 and display unit 141 may be configured integral withthe input/output panel 200 as illustrated in FIGS. 3A and 3B.

The input control unit 152 performs control regarding operation inputreceived by the operation reception unit 151.

The UI control unit 160 controls the display unit 141 to display variousimages for executing a function that the user intends. For example, theUI control unit 160 controls the display unit 141 to display anoperation assistance image for assisting imaging operation of variousmenu screens or a panoramic image.

The playback control unit 170 performs playback control at the time ofplaying an image file stored in the recording medium 180. For example,the playback control unit 170 supplies an image file stored in therecording medium 180 to the image processing unit 130 via the temporarystorage unit 114 to perform image processing such as image decodingprocessing and so forth, and displays the image subjected to the imageprocessing on the display unit 141. Also, the playback control unit 170controls an image playback order or display method at the time ofdisplay thereof.

The recording medium 180 is a recording device for storing the imagesupplied from the image processing unit 130 as an image file. Also, therecording medium 180 supplies a stored image file to the imageprocessing unit 130 under control of the playback control unit 170. Forexample, various types of data such as image data in the JPEG format andso forth are stored in the recording medium 180. Note that the recordingmedium 180 may be built in the imaging apparatus 100, or may bemountable/detachable from the imaging apparatus 100. Also, variousrecording media may be employed as the recording medium 180, such assemiconductor memory, an optical recording medium, a magnetic disk, anHDD (Hard Disk Drive), and so forth. Note that recordable DVD (DigitalVersatile Disk), recordable CD (Compact Disc), Blu-ray Disc (registeredtrademark), and so forth may be used as an optical recording medium, forexample.

Functional Configuration Example of Imaging Apparatus

FIG. 2 is a block diagram illustrating a functional configurationexample of the imaging apparatus 100 according to the first embodimentof the present disclosure. The imaging apparatus 100 includes theimaging unit 112, temporary storage unit 114, orientation detecting unit121, movement amount detecting unit 131, and panoramic image generatingunit 132. Also, the imaging apparatus 100 includes the display unit 141,operation reception unit 151, recording medium 180, control unit 190,display control unit 191, and recording control unit 192. Note that, inFIG. 2, of the units shown in FIG. 1, only portions principally relatingto image processing and display processing at the time of imaging of apanoramic image are illustrated, and other portions are omitted.

The imaging unit 112 images a subject to generate an image under controlof the control unit 190, and supplies the generated image to thetemporary storage unit 114. Also, in the event that the panoramic imageimaging mode has been set, the imaging unit 112 consecutively images asubject in chronological order to generate multiple images, and suppliesthe generated images to the temporary storage unit 114.

The temporary storage unit 114 is image memory in which the imagesgenerated by the imaging unit 112 are held, and supplies the held imagesto the movement amount detecting unit 131 and panoramic image generatingunit 132.

The orientation detecting unit 121 detects change in the orientation ofthe imaging apparatus 100, and outputs orientation change informationrelating to the detected orientation change to the control unit 190. Forexample, the orientation detecting unit 121 detects change in theorientation of the imaging apparatus 100 which changes according to therotational movement of the imaging apparatus 100 in a particulardirection (e.g., horizontal direction) with a neighborhood position ofthe imaging apparatus 100 (e.g., backward position) as an axis.

The movement amount detecting unit 131 detects a movement amount and amovement direction between adjacent images at the time axis regardingthe images held in the temporary storage unit 114, and outputs thedetected movement amount and movement direction to the panoramic imagegenerating unit 132. For example, the movement amount detecting unit 131performs matching processing between pixels making up adjacent twoimages (i.e., matching processing for distinguishing the imaging regionsof the same subject) to calculate the number of pixels moved between theimages. With this matching processing, processing assuming that asubject basically remains stationary is performed. Note that, in theevent that an object is included in a subject, a motion vector differentfrom the motion vector of the entire image is detected, but theprocessing is performed with the motion vector corresponding to such anobject being taken as out of detected objects. That is to say, only themotion vector corresponding to the movement of the entire image whichoccurs along with the movement of the imaging apparatus 100 (GlobalMotion Vector (GMV)) is detected. Note that the movement amountdetecting unit 131 corresponds to the image processing unit 130 shown inFIG. 1.

The panoramic image generating unit 132 generates a panoramic imageusing the multiple images held in the temporary storage unit 114 undercontrol of the control unit 190, and supplies the generated panoramicimage to the display control unit 191 and recording control unit 192.For example, the panoramic image generating unit 132 calculates a regionto be synthesized in each of the multiple images held in the temporarystorage unit 114 based on the analysis results output from the controlunit 190 (analysis results of the change amount of the orientation ofthe imaging apparatus 100). The panoramic image generating unit 132extracts an image from the region to be synthesized of each of themultiple images, and synthesizes the extracted images to generate apanoramic image. In this case, the panoramic image generating unit 132synthesizes the extracted images thereof in an overlapped manner basedon the movement amount and movement direction output from the movementamount detecting unit 131 to generate a panoramic image. Also, thepanoramic image generating unit 132 sequentially supplies an image fordisplaying an image generated by the imaging unit 112 and held in thetemporary storage unit 114 as a through image to the display controlunit 191. Note that the panoramic image generating unit 132 correspondsto the image processing unit 130 shown in FIG. 1.

The operation reception unit 151 is an operating unit for receivingoperation input operated by the user, and outputs an operation signalcorresponding to the received operation input to the control unit 190.

The control unit 190 controls the units of the imaging apparatus 100based on the operation signal from the operation reception unit 151, andthe orientation change information from the orientation detecting unit121. For example, in the event of an imaging mode setting operationbeing received at the operation reception unit 151, the control unit 190sets the imaging mode corresponding to the setting operation thereof.Note that, with the first embodiment of the present disclosure,description will be made regarding a case where a first imaging modethrough a third imaging mode (illustrated in FIG. 7A) are set as thepanoramic image imaging mode for generating a panoramic image, as anexample. Also, in the event that a setting operation for the referenceposition (shown in FIG. 7B) has been received by the operation receptionunit 151, the control unit 190 sets the reference position correspondingto the setting operation thereof.

Also, the control unit 190 determines a subject to be situated in thereference position in a panoramic image based on the operation signalfrom the operation reception unit 151. Specifically, the control unit190 determines a subject that has been an imaging object of the imagingunit 112 at the time of a determination instructing operation (e.g., apressing operation of the shutter button 210) by the user beingreceived, to be a subject to be situated in the reference position. Forexample, in the event that the first imaging mode or second imaging modehas been set, the control unit 190 determines a subject to be situatedin the reference position based on the user's operations before start ofthe panoramic image imaging operation. Also, in the event that the thirdimaging mode has been set, the control unit 190 determines a subject tobe situated in the reference position based on the user's operationsduring the panoramic image imaging operation. For example, in the eventthat the first imaging mode has been set, the control unit 190determines the rotation angle of the imaging apparatus 100 at the timeof start of imaging operation with the orientation of the imagingapparatus 100 as a reference at the time of a determination instructingoperation of a subject to be situated in the reference position beingreceived, to be a start position. Also, in the event that one of thefirst though third imaging modes has been set, the control unit 190determines the rotation angle of the imaging apparatus 100 at the timeof end of the imaging operation with the orientation of the imagingapparatus 100 at the time of a determination instructing operationthereof being received as a reference, to be an end position.

Also, the control unit 190 performs imaging control of the imaging unit112 based on the orientation change information output from theorientation detecting unit 121 so that the subject determined to asubject to be situated in the reference position in a panoramic image issituated in the reference position thereof. Specifically, in the eventthat the first imaging mode has been set, the control unit 190determines the start position and end position of the panoramic imageimaging operation so that the determined subject thereof is situated inthe reference position. The control unit 190 then determines whether theimaging apparatus 100 having arrived at the start position or endposition based on the orientation change information thereof, starts theimaging operation at the time of the imaging apparatus 100 havingarrived at the start position, and ends the imaging operation at thetime of the imaging apparatus 100 having arrived at the end position.

Also, in the event that the second imaging mode has been set, thecontrol unit 190 starts the imaging operation thereof based on theuser's operations for instructing start of the panoramic image imagingoperation, and determines the end position of the imaging operationthereof so that the determined subject thereof is situated in thereference position. The control unit 190 then determines whether or notthe imaging apparatus 100 has arrived at the end position based on theorientation change information thereof, and ends the imaging operationat the time of the imaging apparatus 100 arriving at the end position.

Also, in the event that the third imaging mode has been set, the controlunit 190 determines the end position of the panoramic image imagingoperation based on the start position of the panoramic image imagingoperation, and the orientation of the imaging apparatus 100 at the timeof determining a subject to be situated in the reference position in apanoramic image. In this case, the end position of the imaging operationthereof is determined so that the determined subject is situated in thereference position. The control unit 190 then determines whether or notthe imaging apparatus 100 having arrived at the end position based onthe orientation change information, and ends the imaging operation atthe time of the imaging apparatus 100 arriving at the end position.

Also, in the event that the user's operations for determining a subjectto be situated in the reference position in a panoramic image have beenreceived, the control unit 190 performs control for displaying anoperation assistance image for changing the orientation of the imagingapparatus 100 to the start position of the panoramic image imagingoperation, on the display unit 141. Also, after the orientation of theimaging apparatus 100 changes to the start position thereof, the controlunit 190 performs control for displaying an operation assistance imagefor changing the orientation of the imaging apparatus 100 to the endposition of the panoramic image imaging operation, on the display unit141.

Also, for example, the control unit 190 analyses the change amount(movement direction, movement amount, etc.) of the orientation of theimaging apparatus 100 based on the orientation change information outputfrom the orientation detecting unit 121, and outputs the analysisresults thereof to the panoramic image generating unit 132 and displaycontrol unit 191. Also, for example, after the generation processing ofmultiple images by the imaging unit 112 is ended, the control unit 190performs control for displaying the panoramic image generated by thepanoramic image generating unit 132 on the display unit 141. Note thatthe control unit 190 is an example of a determining unit and controlunit described in the Summary.

The display control unit 191 displays the panoramic image generated bythe panoramic image generating unit 132 on the display unit 141 undercontrol of the control unit 190. For example, in the event that theuser's operations for determining a subject to be situated in thereference position in a panoramic image have been received, the displaycontrol unit 191 displays an operation assistance image for changing theorientation of the imaging apparatus 100 to the start position of thepanoramic image imaging operation, on the display unit 141. Also, afterthe orientation of the imaging apparatus 100 is changed to the startposition thereof, the display control unit 191 displays an operationassistance image for changing the orientation of the imaging apparatus100 to the end position of the panoramic image imaging operation, on thedisplay unit 141.

The display unit 141 is a display unit for displaying the image suppliedfrom the display control unit 191. Also, various menu screens andvarious images are displayed on the display unit 141.

The recording control unit 192 performs control for recording thepanoramic image generated by the panoramic image generating unit 132 inthe recording medium 180 under control of the control unit 190.

The recording medium 180 stores the panoramic image generated by thepanoramic image generating unit 132 as an image file (image content).

Appearance Configuration Example of Imaging Apparatus

FIGS. 3A and 3B are perspective views illustrating the appearance of theimaging apparatus 100 according to the first embodiment of the presentdisclosure. FIG. 3A is a perspective view illustrating the appearance onthe front (i.e., face where the lens to be directed to a subject isprovided) side of the imaging apparatus 100. Also, FIG. 3B is aperspective view illustrating the appearance on the back (i.e., facewhere the input/output panel 200 to be directed to a subject isprovided) side of the imaging apparatus 100.

The imaging apparatus 100 includes the lens unit 111, shutter button210, and input/output panel 200. The imaging apparatus 100 is an imagingapparatus whereby a subject can be imaged to generate an image (imagedata), and this generated image can be recorded in the recording medium180 (shown in FIG. 1) as an image content (still image content or movingimage content). Note that other operating members such as a powerswitch, a mode changeover switch, a zoom button, and so forth areprovided to the imaging apparatus 100, but drawing and descriptionthereof will be omitted.

The shutter button 210 is a button to be pressed by the user at the timeof recording an image (image data) generated by imaging a subject as animage content. For example, in the event that a still image imaging modefor recording a still image has been set, when the shutter button 210 ishalf-pressed, focus control for performing autofocus is performed. Also,when the shutter button 210 is full-pressed, focus control thereof isperformed, and the image imaged by the imaging unit 112 at the time ofthis full-pressing is recorded in the recording medium 180 in acorrelated manner. Also, the pressing operation of the shutter button210 in the event that the panoramic image imaging mode has been set willbe described in detail with reference to FIGS. 8A through 23B andothers.

The input/output panel 200 displays various images, and also receivesoperation input from the user by detecting a contact operation on theinput/output panel 200. The input/output panel 200 is realized by atouch panel, for example. Note that the input/output panel 200corresponds to the display unit 141 and operation reception unit 151shown in FIGS. 1 and 2.

Display Screen Example when Setting Panoramic Image Imaging Mode

FIG. 4 is a diagram illustrating a display screen example to bedisplayed on the input/output panel 200 according to the firstembodiment of the present disclosure. With the display screen 300 shownin FIG. 4, icons (301 through 305) displayed on the left side are iconsprincipally used at the time of performing setting operations, and icons(306 through 309) displayed on the right side are icons principallyrepresenting the current state and so forth.

A transition button 301 for a menu screen is a button to be pressed atthe time of making a transition to the menu screen. Specifically, uponthe transition button 301 for the menu screen being pressed, the menuscreen is displayed on the input/output panel 200.

A transition button 302 for a moving image imaging mode is an icon to bepressed at the time of setting the moving image imaging mode.Specifically, upon the transition button 302 for the moving imageimaging mode being pressed, the moving image imaging mode is set, andimaging operation of a moving image (taking of a moving image) may beperformed.

A swing direction setting button 303 is a button to be pressed at thetime of setting a swing direction in the event that the panoramic imageimaging mode has been set. Also, a marker corresponding to a swingdirection that has been set is displayed as a swing direction settingbutton 303. FIG. 4 illustrates a case where a setting has been set sothat a swing operation is performed from the left to the right. Notethat, a swing direction setting screen 320 to be displayed in the eventthat the swing direction setting button 303 has been pressed will bedescribed in detail with reference to FIG. 5A. Note that, with the firstembodiment of the present disclosure, description will be made withreference to the rotation angle of the imaging apparatus 100 in aparticular direction (e.g., horizontal direction) with the currentposition of the imaging apparatus 100 (a neighborhood position of theimaging apparatus 100 (e.g., backward position)) being taken as an axis,as a swing operation. Also, the operation direction thereof will bereferred to as swing direction.

An image size setting button 304 is a button to be pressed at the timeof setting an image size in the swing direction in the event that thepanoramic image imaging mode has been set. With the first embodiment ofthe present disclosure, an example is shown wherein standard (180degrees) and wide (240 degrees) are set as an image size. Also, a markercorresponding to the image size set as the image size setting button 304is displayed. FIG. 4 illustrates a case where standard (STD) has beenset. Also, the swing direction setting button 303 and image size settingbutton 304 are displayed only in the event that the panoramic imageimaging mode has been set. Note that an image size setting screen 330 tobe displayed when the image size setting button 304 is pressed will bedescribed in detail with reference to FIG. 5B.

An exposure value setting button 305 is a button to be pressed at thetime of setting an exposure value (EV value). Specifically, upon theexposure value setting button 305 being pressed, a setting screen forsetting an exposure value is displayed.

A number-of-recordable panoramic images informing icon 306 is an iconrepresenting the number of panoramic images which can be recorded bymemory (e.g., recording medium 180) housed in the imaging apparatus 100.For example, as the number-of-recordable panoramic images informing icon306, a marker schematically representing memory, and a number “10”(situated on the lower right portion of the marker representing memory)indicating the number of recordable panoramic images are displayed.

A panoramic image size informing icon 307 is an icon representing theimage size in the swing direction in the event that the panoramic imageimaging mode has been set. Specifically, a marker representing the imagesize set after pressing of the image size setting button 304 isdisplayed.

A setting mode informing icon 308 is an icon representing the currentlyset mode. For example, a marker representing one of the playback mode,and imaging mode (still image imaging mode, panoramic image imagingmode, and moving image imaging mode). Note that FIG. 4 illustrates amarker in the event that the panoramic image imaging mode has been setas the setting mode informing icon 308.

A transition button 309 for the playback mode is a button to be pressedat the time of changing to the playback mode. Specifically, upon thetransition button 309 for the playback mode being pressed, the playbackmode is set.

A focus frame 310 is a frame for specifying an object to be focused(focused object) of subjects displayed on the input/output panel 200,and is situated near the center of the display screen 300 as four whitebrackets. Specifically, of subjects included in the frame of the focusframe 310, one or multiple subjects are focused.

A panorama imaging operation operational assistance image 311 is anoperational assistance image for informing the user of the swingdirection. Specifically, the panorama imaging operation operationalassistance image 311 is displayed in the event that the panoramic imageimaging mode has been set, and is made up of five cameras virtuallysituated in the shooting position of a panoramic image. Also, thepanorama imaging operation operational assistance image 311 informs theswing direction set in the panoramic image imaging mode (swing directionset after pressing the swing direction setting button 303) by displayswitching of the five cameras. For example, let us assume a case where asetting for swinging the imaging apparatus 100 from the left to theright has been set. In this case, one of the five cameras making up thepanorama imaging operation operational assistance image 311 issequentially switched to a particular color toward from the left to theright. After the right-edge camera of the five cameras is switched to aparticular color, the left-edge camera is switched to a particularcolor. Hereafter, switching display of a camera to a particular color iscontinuously performed in the same way. FIG. 4 illustrates a state inwhich the camera in the middle is set to a particular color. Note thatthe panorama imaging operation operational assistance image 311 iseliminated when a start instructing operation (shutter pressingoperation) for starting the panoramic image imaging operation isperformed.

A message display region 312 is a region for displaying a message forassisting operations as to the user.

Notification information 313 is a region for notifying the user of asubject portion that may not be included in a panoramic image in theevent that the panoramic image imaging mode has been set. Specifically,the notification information 313 is set to be darkish overall, whereby asubject portion that may not be included in a panoramic image can benotified to the user. For example, in the initial state of the swingoperation in the panoramic image imaging mode, with regard to a subjectequivalent to the left edge of the display screen 300, the image thereofis not generated, and is not included in the panoramic image. Therefore,as shown in FIG. 4, in the initial state of the swing operation in thepanoramic image imaging mode, the notification information 313 isdisplayed on the left edge of the display screen 300. Also, according tothe swing operation in the panoramic image imaging mode, thenotification information 313 is moved from the left side to the rightside. In this case, the size of the width of the notificationinformation 313 is changed according to the movement thereof.

Setting Screen Example when Setting Swing Direction and Image Size

FIGS. 5A and 5B are diagrams illustrating a setting screen example to bedisplayed on the input/output panel 200 according to the firstembodiment of the present disclosure. FIG. 5A illustrates the swingdirection setting screen 320. The swing direction setting screen 320 is,with the display screen 300 shown in FIG. 4, a screen to be displayedoverlapped with the display screen 300 after the swing direction settingbutton 303 is pressed.

With the swing direction setting screen 320, swing direction selectingbuttons 321 through 324, a “x” button 325 and a “?” button 326 areprovided.

The swing direction selecting buttons 321 through 324 are buttons to bepressed at the time of a direction specified by an arrow appended withineach of the markers being selected as the swing direction. For example,in the event of setting to perform the swing operation from the left tothe right, the swing direction setting button 321 is pressed. Also, inthe event that a selection operation by the user has been performed, aframe 327 to the effect that is appended.

The “x” button 325 is a button to be pressed at the time of eliminatingthe swing direction setting screen 320, and the “?” button 326 is abutton to be pressed at the time of using a help function.

FIG. 5B illustrates the image size setting screen 330. The image sizesetting screen 330 is a screen to be displayed overlapped with thedisplay screen 300 after the image size setting button 304 is pressedwith the display screen 300 shown in FIG. 4.

With the image setting screen 330, the image size selecting buttons 331and 332, “x” button 333, and “?” button 334 are provided.

The image size selecting buttons 331 and 332 are buttons to be pressedat the time of selecting the image size specified by a letter appendedin each of the markers. For example, in the event of selecting standard(180 degrees), the image size selecting 331 is pressed, and in the eventof selecting wide (240 degrees), the image size selecting button 332 ispressed. Also, in the event that a selection operation by the user hasbeen performed, a frame 335 to the effect that is appended. Note that,in this example, an example is illustrated wherein the user selects adesired image size out of the two kinds of image sizes, but the user mayselect a desired image size out of three or more kinds of image sizes.Alternatively, the image size may be set by input operation by the user(e.g., input operation of the numeric value of an image size (angle)).

The “x” button 333 is a button to be pressed at the time of eliminatingthe image size setting screen 330, and the “?” button 334 is a button tobe pressed at the time of using the help function.

Imaging Operation Example of Panoramic Image

Next, description will be made regarding a case where imaging operationis performed using the imaging apparatus 100.

FIGS. 6A and 6B are diagrams illustrating a relation between imagingoperation in the event of generating a panoramic image using the imagingapparatus 100 according to the first embodiment of the presentdisclosure, and a panoramic image generated by this imaging operation.

FIG. 6A illustrates a top view when a place 400 serving as an imagingobject is viewed from above in the event of generating a panoramic imageusing the imaging apparatus 100. The place 400 is a place includingmountains, buildings, trees, a lake, a cattle image 401, and so forth,and in FIG. 6A, in order to simplify description, these are illustratedin a simplified manner. Also, FIG. 6A illustrates an example wherein aswing operation from the left to the right is set as the swingdirection, and standard (180 degrees) is set as the image size. Also,FIG. 6A illustrates an example wherein the user operates the imagingapparatus 100 to generate a panoramic image so that the position of thecattle image 401 becomes the center position of the panoramic image.

FIG. 6B illustrates a panoramic image 410 generated by the imagingoperation shown in FIG. 6A. Specifically, the panoramic image 410 is apanoramic image where the cattle image 401 is situated in the centerposition. In this way, this assumes a case where a user 404 who wandersaround the lakeside desires a panoramic image where the cattle image 401is situated at the center position. In this case, for example, the user404 has to start the panoramic image imaging operation at the startposition 402 of the imaging operation, and has to end the panoramicimage imaging operation at the end position 403 of the imagingoperation.

In order to perform imaging operation in this way, the user 404 has tolook over the circumference in the place 400, recognize the startposition 402 and end position 403, and start imaging operation. However,it is assumed to be difficult for the user 400 to look over thecircumference in the place 400, and to accurately recognize the startposition 402 and end position 403 by visual observation. Also, in theevent that it is difficult to accurately recognize the start position402 and end position 403, it is difficult to generate a panoramic imagethat the user 404 desires (panoramic image where the cattle image 401 issituated in the center position). In this case, the user 404 has toselect whether imaging operation is performed again after confirming thestart position 402 and end position 403 again, or the user 404 satisfiesa panoramic image that the user 404 does not intend (i.e., panoramicimage where the cattle image 401 is not situated in the centerposition).

Therefore, with the first embodiment of the present disclosure, anexample will be shown wherein the image control of the imaging unit 112is performed so that a subject determined to be a subject to be situatedin the reference position in a panoramic image is situated in thereference position.

Setting Screen Example of Imaging Conditions

FIGS. 7A and 7B are diagrams illustrating a setting screen example to bedisplayed on the input/output panel 200 according to the firstembodiment of the present disclosure. FIG. 7A illustrates the panoramicimage imaging mode setting screen 340. The panoramic image imaging modesetting screen 340 is a screen to be displayed on the input/output panel200 at the time of setting the type of the panoramic image imaging mode.For example, the panoramic image imaging mode setting screen 340 isdisplayed immediately after a setting operation of the panoramic imageimaging mode for recording a panoramic image is performed. With thepanoramic image imaging mode setting screen 340, first through fourthimaging mode setting buttons 341 through 344, a determination button345, and a return button 346 are provided.

The first through fourth imaging mode setting buttons 341 through 344are buttons to be pressed when setting the type of the panoramic imageimaging mode. The first imaging mode setting button 341 is a button tobe pressed when setting the first imaging mode. Note that the firstimaging mode will be described in detail with reference to FIGS. 8Athrough 14B. The second imaging mode setting button 342 is a button tobe pressed when setting the second imaging mode. Note that the secondimaging mode will be described in detail with reference to FIGS. 16Athrough 19B. The third imaging mode setting button 343 is a button to bepressed when setting the third imaging mode. Note that the third imagingmode will be described in detail with reference to FIGS. 20A through23B. The fourth imaging mode setting button 344 is a button to bepressed when setting the fourth imaging mode. Note that the fourthimaging mode will be described in detail in the second embodiment of thepresent disclosure.

The determination button 345 is a button to be pressed when a pressingoperation for selecting one of the first through fourth imaging modes isperformed, and then selection thereof is determined. Also, informationrelating to the determined imaging mode by the pressing operation of thedetermination button 345 (imaging mode information) is output to thecontrol unit 190 for holding. The return button 346 is a button to bepressed in the event of returning to the display screen immediatelybefore the current screen, for example.

FIG. 7B illustrates a reference position setting screen 350. Thereference position setting screen 350 is a screen to be displayed on theinput/output panel 200 when setting the reference position in apanoramic image. For example, the reference position setting screen 350is displayed immediately after the determination button 350 is pressedwith the panoramic image imaging mode setting screen 340 shown in FIG.7A. With the reference position setting screen 350, a reference positionspecifying bar 351, a specified position 352, a determination 353, and areturn button 354 are provided.

The reference position specifying bar 351 is a bar used for setting thereference position in a panoramic image, and displayed overlapped withthe specified position 352. For example, with the reference positionspecifying bar 351, the user moves the reference position specifying bar351 to a desired reference position, whereby the reference position in apanoramic image can be specified. For example, in the event thatstandard (180 degrees) has been set as the image size, “−90°” isdisplayed on one of the edges of the reference position specifying bar351, and “90°” is displayed on the other edge. Also, with the referenceposition specifying bar 351, “−60°”, “−30°”, “0°”, “30°”, and “60°” aredisplayed with a certain interval. Here, the reference position meansthe position in a panoramic image where the user intends to situate adesired subject. For example, in the event that the horizontal directionhas been set as the swing direction, a position in the horizontaldirection in a panoramic image becomes the reference position.

Also, this example shows an example wherein the reference position in apanoramic image is set by the user's operations, but the referenceposition thereof may be set beforehand. For example, the center position(i.e., position of 0°) may be set as the reference position.

The determination button 353 is a button to be pressed when a specifyingoperation for specifying the reference position in a panoramic image isperformed, and then specification thereof is determined. Also,information relating to the determined reference position by thepressing operation of the determination button 353 (reference positioninformation) is output to the control unit 190 for holding. The returnbutton 354 is a button to be pressed in the event of returning to thedisplay screen immediately before the current screen, for example.

Imaging Control Example in First Imaging Mode

FIGS. 8A through 14B are diagrams schematically illustrating atransition example of the orientation of the imaging apparatus 100according to the first embodiment of the present disclosure, and adisplay screen displayed on the input/output panel 200. Specifically, atransition example of the orientation of the imaging apparatus 100 isillustrated in FIGS. 8A, 9A, 10A, 11A, 12A, 13A, and 14A, and atransition example of a display screen displayed on the input/outputpanel 200 according to the change in the orientation of the imageapparatus 100 is illustrated in FIGS. 8B, 9B, 10B, 11B, 12B, 13B, and14B. Also, FIGS. 8A through 14B illustrate an imaging operation examplein the event that the first imaging mode has been set, a swing operationfrom the left to the right has been set as the swing direction, andstandard (180 degrees) has been set as the image size. Also, FIGS. 8Athrough 14B illustrate an imaging operation example in the event thatthe center position (i.e., the position of “0°” on the referenceposition specifying bar 351) has been set as the reference position in apanoramic image.

FIG. 8A illustrates a case where the imaging apparatus 100, and a user415 holding the imaging apparatus 100 in both hands are viewed fromabove, in a simplified manner. In this case, let us say that the user415 confirms the reference position (center position) in a panoramicimage while viewing a subject displayed on the input/output panel 200.

FIG. 8B illustrates the display screen 300 to be displayed before adetermination operation for determining a subject to be situated in thereference position in a panoramic image is performed. The display screen300 is the same as the display screen 300 shown in FIG. 4, wherein“PRESS SHUTTER BUTTON WHILE DIRECTING LENS TO REFERENCE POSITION INPANORAMIC IMAGE” is displayed in the message display region 312. Also,with the display screen 300, a subject that is the current imagingobject is displayed as a through image. Also, with the first embodimentof the present disclosure, an example is shown wherein a startinstructing operation for imaging the operation of a panoramic image,and a determining operation of a subject to be situated in the referenceposition in the first imaging mode are performed by a pressing operationof the shutter button 210.

Specifically, in the event that the first imaging mode has been set, theuser 415 may start imaging operation without accurately recognizing thestart position and end position in a panoramic image by determining asubject to be situated in the reference position in the panoramic image.

FIG. 9A illustrates a case where the user 415 has pressed the shutterbutton 210 in the state shown in FIG. 8A, in a simplified manner. Also,in this example, let us say that the image size that has been set is θ3(=180 degrees), and an angle from the reference position 421 to thestart position 422 of the imaging operation is θ1 (=90 degrees), and anangle from the reference position 421 to the end position 423 of theimaging operation is θ2 (=90 degrees). Note that θ1 and θ2 are valuesthat are determined according to the reference position. For example, inthe event that the center position has been set as the referenceposition, θ1=θ3/2 and θ2=θ3/2 hold. Also, let us assume a case wherestandard (180 degrees) has been set as the image size, and a positionother than the center position has been set as the reference position(e.g., the position of “30°” in the reference position specifying bar351). In this case, θ1=θ3×(⅔) and θ2=θ3×(⅓) hold.

For example, let us assume a case where the user has pressed the shutterbutton 210 in the state shown in FIG. 8A. In this case, the startposition 422 and end position 423 of the imaging operation is determinedbased on the image size θ3 that has been set with the optical-axisdirection from a position 420 at the time of pressing thereof (thedirection of the reference position 421) being taken as a reference.Specifically, as shown in FIG. 9A, the position 420 at the time ofpressing the shutter button 210 is taken as the rotation axis, and aposition rotated by the angle θ1 in a direction opposite from the swingdirection is determined to be the start position 422 of the imagingoperation. Similarly, the position 420 at the time of pressing theshutter button 210 is taken as the rotation axis, and a position rotatedby the angle θ2 in the swing direction is determined to be the endposition 423 of the imaging operation. Note that the user 415 and theimaging apparatus 100 are indicated with a dotted line in FIG. 9A.

FIG. 9B illustrates a display screen 360 to be displayed at the time ofa determination operation for determining the reference position in apanoramic image being performed. The display screen 360 is a displayscreen wherein, with the display screen shown in FIG. 4, the focus frame310 and panorama imaging operation operational assistance image 311 areeliminated, and an arrow 361 is displayed. Also, “SWING IMAGINGAPPARATUS IN ARROW DIRECTION UP TO START POSITION” is displayed in themessage region 362. In this way, after pressing the shutter button 210,the display screen 360 is displayed on the input/output panel 200,whereby the user 415 can readily move the imaging apparatus 100 to thestar position 422 in accordance with the display screen 360. Also, adisplay screen example that is displayed on the input/output panel 200after the user 415 starts a swing operation will be illustrated in FIG.10B and others.

FIG. 10A illustrates a case where the user 415 has moved the imagingapparatus 100 in the opposite direction of a swing direction 424 in thestate shown in FIG. 9A, in a simplified manner.

FIG. 10B illustrates a swing operation assistance screen 370 to bedisplayed on the input/output panel 200 after the user 415 moves theimaging apparatus 100 in an arrow 425 direction. Note that the movementof the imaging apparatus 100 after pressing the shutter button 210 isdetermined by the control unit 190 based on the orientation changeinformation from the orientation detecting unit 121.

With the swing operation assistance screen 370, a swing operationassistance image display region 371, an arrow 373, and a message displayregion 374 are provided. Also, with the swing operation assistancescreen 370, a focusing informing icon 375, a shutter speed informingicon 376, an F value informing icon 377, and a flash prohibition icon378 are provided.

The swing operation assistance image display region 371 is a bar graphfor informing the user of the user's operation relating to the panoramicimage imaging operation (swing operation of the imaging apparatus 100).Specifically, with the swing operation assistance image display region371, a current operation position 372 as to the entire operation amountfor the panoramic image imaging mode (e.g., rotation speed of a swingoperation) is displayed. For example, with regard to the swing operationassistance image display region 371, based on the detection results of amovement amount and a movement direction between adjacent images at thetime axis, the control unit 190 calculates the current operation amount,whereby a display state thereof can be changed based on this currentoperation amount. As a movement amount and a movement direction thereof,a motion vector corresponding to the motion of the entire image thatoccurs along with the movement of the imaging apparatus 100 (GlobalMotion Vector (GMV)) is detected. Also, based on the angular velocitydetected by the orientation detecting unit 121, the control unit 190 maycalculate the current operation amount. Alternatively, the control unit190 may calculate the current operation amount using the detectionresults of the movement amount and movement direction thereof, and theangular velocity detected by the orientation detecting unit 121. In thisway, the user can readily recognize how much longer the user has toperform the swing operation by displaying the swing operation assistanceimage display region 371 in this way.

Also, the operation position 372 is displayed with blinking in thecenter position in the swing operation assistance image display region371 immediately after the user 415 starts the swing operation. Also, theoperation position 372 that is displayed with blinking advances in theleft direction according to the swing operation of the imaging apparatus100 by the user (swing operation in the arrow 425 direction). Theoperation position 372 is displayed with blinking until the imagingapparatus 100 arrives at the start position 422 by the swing operationof the user 415. The operation position 372 is displayed with blinkingin this way, thereby informing the user that this movement is not duringthe panoramic image imaging operation, but a movement for preparationthereof. Also, in the event that the imaging apparatus 100 has arrivedat the start position 422, and a start instructing operation of thepanoramic image imaging operation by the user 415 has been performed,the display mode of the operation position 372 is changed from blinkingdisplay to normal display (constant lighting). The display mode of theoperation position 372 is changed in this way, whereby the user canreadily recognize whether the current state is during preparation forthe panoramic image imaging operation, or during the panoramic imageimaging operation.

Also, with the center position in the swing operation assistance imagedisplay region 371, a reference position mark 379 representing thereference position is displayed. With this example, a dotted line isappended to the swing operation assistance image display region 371 asthe reference position mark 379. Also, with this example, the image sizethat has been set is 180 degrees, and the center position has been setas the reference position, and accordingly, the reference position mark379 is appended to the center position in the swing operation assistanceimage display region 371. For example, let us assume a case where aposition other than the center position (e.g., the position of “30°” inthe reference position specifying bar 351) has been set. In this case,the reference position mark 379 is appended to a right-wards position inthe swing operation assistance image display region 371 (position of 4/6(=(90°+30°)/180°) from the left edge in the event that the length in thelongitudinal direction is taken as 1).

The arrow 373 is an arrow for assisting the user's operations relatingto the panoramic image imaging operation (the swing operation of theimaging apparatus 100). That is to say, the user swings the imagingapparatus 100 in a direction that the arrow 373 specifies, whereby thepanoramic image imaging operation, and preparation thereof can beperformed.

The message display region 374 is a region where a message for assistingthe user's operations relating to the panoramic image imaging operation(the swing operation of the imaging apparatus 100) is displayed.

The focusing informing icon 375 is an icon for informing the userwhether or not a subject positioned in a focus frame comes into focus.Specifically, in the event that the subject comes into focus, thefocusing informing icon 375 is changed to green, and in the event thatthe subject does not come into focus, the green thereof is blinked.

The shutter speed informing icon 376 is an icon for informing the userof the shutter speed that has currently been set. For example, as shownin FIG. 10B, in the event that “20” is displayed as the shutter speedinforming icon 376, this means that the shutter speed that has currentlybeen set is “ 1/20”.

The F value informing icon 377 is an icon for informing the user of an Fvalue that has currently be set. For example, as shown in FIG. 10B, inthe event that “F3.5” is displayed as the F value informing icon 377,this means that the F value that has currently been set is “F3.5”.

The flash prohibition icon 378 is an icon for informing the user thatusing the flash is prohibited in the imaging mode that has currentlybeen set. Specifically, only in the event that using the flash isprohibited in the imaging mode that has currently been set, the flashprohibition icon 378 is displayed, and in the event that using the flashis not prohibited in the imaging mode that has currently been set, theflash prohibition icon 378 is not displayed. For example, in the eventthat the panoramic image imaging mode has been set, using the flash isprohibited, and accordingly, the flash prohibition icon 378 isdisplayed.

FIG. 11A illustrates a case where the user 415 has moved the imagingapparatus 100 to the start position 422 of the imaging operation, in asimplified manner.

FIG. 11B illustrates a swing operation assistance screen 380 to bedisplayed on the input/output panel 200 in the event that the user 415has moved the imaging apparatus 100 to the start position 422 of theimaging operation. Note that, with regard to whether or not the imagingapparatus 100 have arrived at the start position 422, the control unit190 determines based on the orientation change information from theorientation detecting unit 121.

The swing operation assistance screen 380 differs from the swingoperation assistance screen 370 shown in FIG. 10B in that the operationposition 372 in the swing operation assistance image display region 371is displayed with blinking on the left edge, and the arrow 373 inverselyfaces. Also, “PRESS SHUTTER BUTTON, AND THEN SWING IMAGING APPARATUS INARROW DIRECTION” is displayed in the message display region 374, and thenotification information 313 is displayed on the left edge of the swingoperation assistance screen 380.

In this way, in the event that the imaging apparatus 100 has been movedto the start position 422 of the imaging operation, a message to theeffect that the imaging operation is started by swinging the imagingapparatus 100 in the opposite direction of the swing direction so far isdisplayed.

Note that, with this example, as a method for informing the user thatthe imaging apparatus 100 has arrived at the start position 433, anexample for informing the user by a message in the swing operationassistance screen 380, an arrow, and so forth is shown, but anotherinforming method may be employed. For example, an informing method fordisplaying a popup message on the swing operation assistance screen 380,and an informing method for displaying the relative position of theimaging apparatus 100 (e.g., a schematic view shown in FIG. 11A) on theswing operation assistance screen 380 may be employed. Also, forexample, an informing method according to audio (e.g., notificationaccording to system sound) may be employed. As for an informing methodaccording to audio, for example, audio “beep, beep, beep.” may beshortened in interval as the imaging apparatus 100 approaches the startposition 422 (e.g., “beep-beep-beep”). In this case, an informing methodby outputting another audio (e.g., “ARRIVED AT START POSITION!”) may beemployed when arriving at the start position 422.

In this way, after the imaging apparatus 100 is moved to the startposition 422 of the imaging operation, and the swing operationassistance screen 380 is displayed on the input/output panel 200, uponthe shutter button 210 being pressed by the user 415, the panoramicimage imaging operation is started. Also, upon the panoramic imageimaging operation being started, the operation position 372 is changedfrom blinking display to a constant lighting state. Also, the operationposition 372 moves according to the movement of the imaging apparatus100.

Note that, with this example, a pressing operation of the shutter button210 by the user 415 is taken as a start condition of the panoramic imageimaging operation, but another start condition may be employed. Forexample, detecting the movement of the imaging apparatus 100 in theopposite direction may be taken as a start condition thereof.

In this way, upon the panoramic image imaging operation being started,the operation position 372 moves according to the movement of theimaging apparatus 100. This example will be shown in FIGS. 12A through13B.

FIGS. 12A and 13A illustrate the transition of the imaging apparatus 100which moves between the start position 422 of the imaging operation andthe end position 423 of the imaging operation.

FIGS. 12B and 13B illustrate swing operation assistance images 381 and382 to be displayed while the imaging apparatus 100 moves between thestart position 422 of the imaging operation and the end position 423 ofthe imaging operation. In this way, after the imaging apparatus 100starts the imaging operation from the start position 422 of the imagingoperation, the swing operation assistance images 381 and 382 aredisplayed according to the movement of the imaging apparatus 100 untilthe imaging apparatus 100 arrives at the end position 423 of the imagingoperation. Also, in the event that the imaging apparatus 100 has arrivedat a position equivalent to the reference position 421 at the time ofthis movement, a message such as “CURRENT SUBJECT IS SITUATED IN THECENTER OF PANORAMIC IMAGE” or the like may be displayed in the messagedisplay region 374.

FIG. 14A illustrates a case where the user 415 has moved the imagingapparatus 100 to the end position 423 of the imaging operation, in asimplified manner.

FIG. 14B illustrates a swing operation assistance screen 383 to bedisplayed on the input/output panel 200 at the time of the user 415moving the imaging apparatus 100 to the end position 423 of the imagingoperation. As shown in FIG. 14B, in the event that the imaging apparatus100 has moved to the end position of the imaging operation, a message tothe effect that the imaging operation has been completed is displayed inthe message display region 374.

The swing operation assistance screen 383 differs from the swingoperation assistance screens 381 and 382 shown in FIGS. 12A and 13A inthat the operation position 372 in the swing operation assistance imagedisplay region 371 is displayed on the right edge, and the arrow 373 iseliminated. Also, “PANORAMIC IMAGE IMAGING OPERATION HAS BEEN COMPLETED”is displayed in the message display region 374, and the notificationinformation 313 is displayed on the right edge in the swing operationassistance screen 370.

In this way, in the event that the imaging apparatus 100 has moved tothe end position 423 of the imaging operation, a message to the effectthat the panoramic image imaging operation has been completed isdisplayed. Also, a panoramic image is generated by the panoramic imagegenerating unit 132 using the multiple images generated by the imagingoperation thereof. After the panoramic image imaging operation iscompleted, the generated panoramic image is recorded in the recordingmedium 180 by the recording control unit 192. Also, for example, afterelapse of a certain period since end of the panoramic image imagingoperation, a generated panoramic image thereof may be preview-displayedon the input/output panel 200.

In this way, imaging control is performed with a subject to be situatedin the reference position being taken as a reference, whereby the user'sfavorite panoramic image can readily be generated.

Note that the swing operation by the user may not correctly be performedwhile the imaging apparatus 100 moves from the start position 422 of theimaging operation to the end position 423 of the imaging operation. Inthis case, an arrangement may be made wherein a swing operationassistance screen 385 shown in FIG. 15 is displayed, thereby promptingthe user to correctly perform the swing operation.

FIG. 15 is a diagram illustrating an example of a swing operationassistance screen to be displayed on the input/output panel 200according to the first embodiment of the present disclosure. The swingoperation assistance screen 385 shown in FIG. 15 is a display screen tobe displayed on the input/output panel 200 in the event that the swingoperation by the user is not correctly performed while the imagingapparatus 100 moves from the start position 422 of the imaging operationto the end position 423 of the imaging operation.

The swing operation assistance screen 385 differs from the swingoperation assistance screens 381 and 382 shown in FIGS. 12A and 13A inthat warning display 386 is displayed, and “PERFORM SWING OPERATIONCORRECTLY AND STRAIGHT” is displayed in the message display region 374.Note that in the event that the swing operation by the user has notcorrectly been performed, the user may be prompted to start thepanoramic image imaging operation after returning to the start position422 of the imaging operation again, for example. In this case, forexample, swing operation assistance screens 370 and 380 shown in FIGS.10B and 11B are displayed on the input/output panel 200, therebyenabling the panoramic image imaging operation to be started.

Imaging Control Example in Second Imaging Mode

An example has been shown so far wherein the image size is setbeforehand, the start position and end position of the imaging operationare calculated based on a determination operation for a subject to besituated in the reference position, and the panoramic image imagingoperation is performed based on these start position and end position.Next, an example will be shown wherein the end position (image size) ofthe imaging operation is calculated based on the movement range of theimaging apparatus 100 from the reference position to the start positionof the imaging operation, and the panoramic image imaging operation isperformed based on this end position.

FIGS. 16A through 19B are diagrams schematically illustrating atransition example of the orientation of the imaging apparatus 100according to the first embodiment of the present disclosure, and adisplay screen to be displayed on the input/output panel 200.Specifically, FIGS. 16A, 17A, 18A, and 19A illustrate a transitionexample of the orientation of the imaging apparatus 100, and FIGS. 16B,17B, 18B, and 19B illustrate a transition example of a display screen tobe displayed on the input/output panel 200 according to change in theorientation of the imaging apparatus 100. Also, with FIGS. 16A through19B, an imaging operation example will be shown wherein the secondimaging mode has been set, a setting for performing the swing operationfrom the left to the right as the swing direction has been performed,and standard (180 degrees) has been set as the image size. Note that inthe event that the second imaging mode has been set, the image size isdetermined based on the movement range of the imaging apparatus 100 fromthe reference position to the start position of the imaging operation,and accordingly, the image size has not to be set beforehand.

Also, with FIGS. 16A through 19B, an imaging operation example will beshown in the event that the center position (i.e., the position of “0”on the reference position specifying bar 351) has been set as thereference position in a panoramic image. Note that the examples shown inFIGS. 16A through 19B are examples in FIGS. 8A through 14B which havepartially been modified, and accordingly, portions common to FIGS. 8Athrough 14B will be denoted with the same reference numerals, and a partof these descriptions will be omitted.

First, as shown in FIG. 8A, the user 415 holding the imaging apparatus100 in both hands confirms a subject to be situated in the referenceposition (center position) in a panoramic image while viewing a subjectdisplayed on the input/output panel 200. In this case, as shown in FIG.8B, the display screen 300 is displayed on the input/output panel 200.Also, with the first embodiment of the present disclosure, an examplewill be shown wherein a start instructing operation of the panoramicimage imaging operation, and a determining operation of a subject to besituated in the reference position in the second imaging mode areperformed by a pressing operation of the shutter button 210.

FIG. 16A illustrates a case where the user 415 has pressed the shutterbutton 210 in the state shown in FIG. 8A, in a simplified manner. Inthis case, a position in the optical-axis direction from a position 430at the time of pressing thereof is determined to be a reference position431. Also, in the event that the image size has been set, in the sameway as with the first imaging mode, the start position and end positionof the imaging operation are determined based on the image size thereof,but for convenience of description, description thereof will be omittedhere.

FIG. 16B illustrates a display screen 390 to be displayed at the time ofa determining operation for determining a subject to be situated in thereference position in a panoramic image being performed. The displayscreen 390 is generally the same as the display screen 360 shown in FIG.9B, but differs in that “SWING IMAGING APPARATUS IN ARROW DIRECTION, ANDPRESS SHUTTER BUTTON AT POSITION SERVING AS LEFT EDGE OF PANORAMICIMAGE” is displayed in the message display region 362. In this way,after pressing the shutter button 210, the display screen 390 isdisplayed on the input/output panel 200, whereby the user 415 canreadily move the imaging apparatus 100 to a position serving as the leftedge of the panoramic image in accordance with the display screen 390.Also, a display screen to be displayed on the input/output panel 200after the user 415 starts a swing operation will be shown in FIG. 17Band others.

FIG. 17A illustrates a case where the user 415 has moved the imagingapparatus 100 in the opposite direction of the swing direction 432(arrow 433 direction) from the state shown in FIG. 16A, in a simplifiedmanner.

FIG. 17B illustrates a swing operation assistance screen 391 to bedisplayed on the input/output panel 200 after the user 415 moves theimaging apparatus 100 in the arrow 433 direction. The swing operationassistance screen 391 is generally the same as the swing operationassistance screen 370 shown in FIG. 10B, but differs in that “SWINGIMAGING APPARATUS IN ARROW DIRECTION, AND PRESS SHUTTER BUTTON ATPOSITION SERVING AS LEFT EDGE OF PANORAMIC IMAGE” is displayed in themessage display region 374. Also, in the event that no image size hasbeen set, the start position and end position of the imaging operationare determined based on a virtual image size (e.g., 180 degrees). Basedon the start position and end position thereof, the current operationposition 372 is displayed in the swing operation assistance imagedisplay region 371.

FIG. 18A illustrates a case where the user 415 has moved the imagingapparatus 100 to a position serving as the left edge of a panoramicimage (start position 434), in a simplified manner. Now, let us assume acase where the user 415 has pressed the shutter button 210 in the stateshown in FIG. 18A. In this case, the end position 435 of the imagingoperation is determined based on the position at the time of pressingthereof (start position 434). Specifically, an angle from the referenceposition 431 to the start position 434 of the imaging operation will betaken as θ4. Also, for example, in the event that an angle from thereference position 431 to the end position 435 of the imaging operationwill be taken as θ5, the center position has been set as the referenceposition, θ4=θ5 holds, and accordingly, an angle θ6 (=2×θ4) isdetermined to be the image size. Also, let us assume a case where aposition other than the center position (e.g., the position of “30°” inthe reference position specifying bar 351) has been set as the referenceposition. In this case, the angle θ6 (=θ4×2−30°×2) is determined to bethe image size.

In this way, in the event that the user 415 has pressed the shutterbutton 210 in the state shown in FIG. 18A, the end position 435 of theimaging operation is determined based on the position at the time ofpressing thereof (start position 434). Note that the user 415 andimaging apparatus 100 are illustrated with a dotted line in FIG. 18A.

FIG. 18B illustrates a swing operation assistance screen 392 to bedisplayed on the input/output panel 200 in the event that the user 415has moved the imaging apparatus 100 to a position serving as the leftedge of the panoramic image (start position 434), and has pressed theshutter button 210. The swing operation assistance screen 392 isgenerally the same as the swing operation assistance screen 380 shown inFIG. 11B, but differs in that “PANORAMIC IMAGE IMAGING OPERATION HASBEEN STARTED, SO SWING IMAGING APPARATUS IN ARROW DIRECTION” isdisplayed in the message display region 374.

In this way, in the event that the imaging apparatus 100 has been movedto the start position of the imaging operation, and the shutter button210 has been pressed, the end position of the imaging operation isdetermined. A message to the effect that the imaging operation isstarted by swinging the imaging apparatus 100 in the opposite directionof the swing direction so far is displayed.

In this way, upon the imaging apparatus 100 being moved to the startposition 434 of the imaging operation, and the shutter button 210 beingpressed by the user 415, the swing operation assistance screen 392 isdisplayed on the input/output panel 200, and the panoramic image imagingoperation is started. Also, upon the panoramic image imaging operationbeing started, the operation position 372 is changed form blinkingdisplay to a constant lighting state. Also, the operation position 372moves according to the movement of the imaging apparatus 100.

Note that, with this example, a pressing operation of the shutter button210 by the user 415 will be taken as a start condition of the panoramicimage imaging operation, but another start condition may be employed.For example, it may be taken as a start condition thereof to detect themovement of the imaging apparatus 100 in the opposite direction. Notethat detection of the movement of the imaging apparatus 100 in theopposite direction is determined by the control unit 190 based on theorientation change information from the orientation detecting unit 121.

Also, the imaging operation after start of the panoramic image imagingoperation is generally the same as with the examples shown in FIGS. 12Athrough 14B, and accordingly, description thereof will be omitted here.

Note that, with the example shown in FIGS. 18A and 18B, an example hasbeen shown wherein an image size less than 180 degrees is determined,but another image size may be determined according to the user'soperations. In FIGS. 19A and 19B, an example is shown wherein an imagesize equal to or greater than 180 degrees is determined.

FIG. 19A illustrates a case where the user 415 has moved the imagingapparatus 100 to a position serving as the left edge of the panoramicimage (start position 436), in a simplified manner. Note that thecalculation method of an image size θ9, the determining method of theend position 437, and so forth are the same as with FIG. 18A.

FIG. 19B illustrates a swing operation assistance screen 393 to bedisplayed on the input/output panel 200 in the event that the user 415has moved the imaging apparatus 100 to a position serving as the leftedge of the panoramic image (start position 436).

In this way, a panoramic image with the user's favorite image size canreadily be generated by the swing operation of the imaging apparatus 100after determination of a subject to be situated in the referenceposition.

Note that, with the first imaging mode and second imaging mode, asubject to be situated in the reference position is determined beforethe imaging apparatus 100 arrives at the start position of the imagingoperation. However, during the imaging operation during the swingoperation from the start position to the end position, a subject to besituated in the reference position may be adjusted by image processing.For example, the panoramic image generating unit 132 obtains and holdsan image (or a part thereof) at the time of determining a subject to besituated in the reference position, and performs matching processingbetween each image during the swing operation from the start position tothe end position and the held image thereof. Based on the results of thematching processing between the held image, and the image equivalent tothe reference position and a position adjacent to this during the swingoperation, the end position may be adjusted so that the determinedsubject is situated in the reference position.

Imaging Control Example of Third Imaging Mode

An example has been shown so far wherein a subject to be situated in thereference position is determined before the panoramic image imagingoperation. Next, an example will be shown wherein a subject to besituated in the reference position is determined after start of theimaging operation, and the end position (image size) of the imagingoperation is calculated based on this determining operation, and thepanoramic image imaging operation is performed based on this endposition.

FIGS. 20A through 23B are diagrams schematically illustrating transitionexamples of the orientation of the imaging apparatus 100 according tothe first embodiment of the present disclosure, and a display screen tobe displayed on the input/output panel 200. Specifically, FIGS. 20A,21A, 22A, and 23A illustrate a transition example of the orientation ofthe imaging apparatus 100, and FIGS. 20B, 21B, 22B, and 23B illustrate atransition example of a display screen to be displayed on theinput/output panel 200 according to change in the orientation of theimaging apparatus 100. Also, FIGS. 20A through 23B illustrate an imagingoperation example in the event that the third imaging mode has been set,a setting for performing the swing operation from the left to the rightas the swing direction has been set, and standard (180 degrees) has beenset as the image size. Note that, in the event that the third imagingmode has been set, the image size is determined based on the referenceposition that is determined after start of the imaging operation, andaccordingly, the image size does not have to be set beforehand.

Also, FIGS. 20A through 23B illustrate an imaging operation example inthe event that the center position (i.e., the position of “0°” on thereference position specifying bar 351) has been set as the referenceposition in a panoramic image. Note that examples shown in FIGS. 20Athrough 23B are the examples in FIGS. 8A through 14B which havepartially been modified, and accordingly, portions common to FIGS. 8Athrough 14B will be denoted with the same reference numerals, and a partof these descriptions will be omitted.

FIG. 20A illustrates a case where the imaging apparatus 100, and theuser 415 holding the imaging apparatus 100 in both hands are viewed fromabove, in a simplified manner. In this case, let us say that the user415 confirms the position of the left edge of a panoramic image whileviewing a subject displayed on the input/output panel 200.

FIG. 20B illustrates a display screen 450 to be displayed before a startinstructing operation of the panoramic image imaging operation. Thedisplay screen 450 is the same as the display screen 300 shown in FIG.4, and “PRESS SHUTTER BUTTON WHILE DIRECTING LENS TO LEFT EDGE POSITIONIN PANORAMIC IMAGE” is displayed in the message display region 312.Also, a subject serving as an imaging object is displayed on the displayscreen 450 by the imaging unit 112 as a through image. Also, with thefirst embodiment of the present disclosure, an example is shown whereina start instructing operation of the panoramic image imaging operation,a determining operation of a subject to be situated in the referenceposition in the third imaging mode are performed by a pressing operationof the shutter button 210.

Specifically, in the event that the third imaging mode has been set, theuser 415 determines a subject to be situated in the reference positionin a panoramic image after start of the imaging operation of thepanoramic image. Thus, the user can perform the imaging operation evenwithout accurately recognizing the end position in the panoramic image.

FIG. 21A illustrates a case where the user 415 has pressed the shutterbutton 210 in the state shown in FIG. 20A. In this case, a position inthe optical-axis direction from the position 440 at the time of pressingthereof is determined to be the start position 441 of the panoramicimage imaging operation. Also, in the event that the image size (e.g.,angle θ10) has been set, in the same way as with the case of the firstimaging mode, the end position 442 of the imaging operation isdetermined based on the image size thereof. Also, in the event that noimage size has been set, the end position of the imaging operation isdetermined based on a virtual image size (e.g., 180 degrees).

FIG. 21B illustrates a swing operation assistance screen 451 to bedisplayed on the input/output panel 200 in the event that the user 415has pressed the shutter button 210. The swing operation assistancescreen 451 is generally the same as the swing operation assistancescreen 380 shown in FIG. 11B, but specifically differs in that“PANORAMIC IMAGE IMAGING OPERATION HAS BEEN STARTED, SO SWING IMAGINGAPPARATUS IN ARROW DIRECTION, AND THEN PRESS SHUTTER BUTTON AT REFERENCEPOSITION” is displayed in the message display region 374. The currentoperation position 372 in the swing operation assistance display region371 is then displayed based on the start position 441 and end position442.

FIG. 22A illustrates a case where the user 415 has determined a subjectto be situated in the reference position by pressing of the shutterbutton 210, in a simplified manner. In this way, in the event that theuser 415 has pressed the shutter button 210 in the state shown in FIG.22A, the end position 445 of the imaging operation is determined basedon the position at the time of pressing thereof (reference position444). Specifically, an angle from the start position 441 to thereference position 444 of will be taken as θ11. Also, for example, inthe event that an angle from the reference position 444 to the endposition 445 will be taken as θ12, the center position has been set asthe reference position, θ11=θ12 holds, and accordingly, an angle θ13(=2×θ11) is determined to be the image size. Also, let us assume a casewhere a position other than the center position (e.g., the position of“30°” in the reference position specifying bar 351) has been set as thereference position. In this case, the angle θ13 (=θ11×2−30°×2) isdetermined to be the image size.

In this way, in the event that the user 415 has pressed the shutterbutton 210 in the state shown in FIG. 22A, the end position 445 of theimaging operation is determined based on the position at the time ofpressing thereof (reference position 444). Note that the user 415 andimaging apparatus 100 are illustrated with a dotted line in FIG. 22A.

FIG. 22B illustrates a swing operation assistance screen 452 to bedisplayed on the input/output panel 200 in the event that the user 415has determined a subject to be situated in the reference position bypressing the shutter button 210. The swing operation assistance screen452 is generally the same as the swing operation assistance screen 380shown in FIG. 11B, but differs in that “REFERENCE POSITION IN PANORAMICIMAGE HAS BEEN DETERMINED. SWING IMAGING APPARATUS IN ARROW DIRECTIONUNTIL IMAGING OPERATION END INSTRUCTION IS ISSUED.” is displayed in themessage display region 374.

In this way, in the event that a subject to be situated in the referenceposition is determined by a pressing operation by the user 415, the endposition of the imaging operation is determined, and a message to theeffect that the imaging operation is continued by swinging the imagingapparatus 100 up to the end position thereof is displayed.

Note that, with this example, a pressing operation of the shutter button210 by the user 415 is taken as a determining method of a subject to besituated in the reference position, but another determining method maybe employed (e.g., a determining method employing another operatingmember).

Also, the imaging operation until end of the panoramic image imagingoperation is generally the same as with the examples shown in FIGS. 12Athrough 14B, and accordingly, description thereof will be omitted here.

Note that, with the example shown in FIG. 22A, an example has been shownwherein a relatively small image size (less than 180 degrees) isdetermined, but another image size may be determined according to theuser's operations. In FIGS. 23A and 23B, an example is shown wherein arelatively great image size (equal to or greater than 180 degrees) isdetermined.

FIG. 23A illustrates a case where the user 415 has determined a subjectto be situated in the reference position by pressing the shutter button210, in a simplified manner. Note that the calculation method of animage size θ16, the determining method of the end position 447, and soforth are the same as with FIG. 22A.

FIG. 23B illustrates a swing operation assistance screen 453 to bedisplayed on the input/output panel 200 in the event that the user 415has determined a subject to be situated in the reference position bypressing the shutter button 210.

In this way, a panoramic image with the user's favorite image size canreadily be generated by the swing operation of the imaging apparatus 100after determination of a subject to be situated in the referenceposition.

As described above, according to the first embodiment of the presentdisclosure, in the event that a panoramic image within a certain rangeis generated, composition determination by the user's visual estimationmay be eliminated. For example, generation of a panoramic image may beperformed by determining a subject to be situated in the referenceposition, whereby repeating of shootings with trial and error to obtaina desired composition can be prevented. That is to say, according to thefirst embodiment of the present disclosure, the user's favoritepanoramic image can readily be generated.

Note that, with the first embodiment of the present disclosure, anexample has been shown wherein the imaging control is performed bydetermining the start position or end position of the panoramic imageimaging operation, but the imaging control may be performed bydynamically trimming image data as to the reference position that hasbeen set. For example, with a system wherein the image size isdetermined beforehand, and a system where there is an upper limitregarding an imaging range, the imaging control may be performed bydynamically trimming image data as to the reference position that hasbeen set so as to generate a panoramic image. In this case, for example,of accumulated image data from the start position of the panoramic imageimaging operation, image data serving as out of an imaging range as tothe reference position is not used. Also, let us assume a case where inthe event that the third imaging mode has been set, the image size orimaging range has reached the upper limit before the user performs adetermining operation of a subject to be situated in the referenceposition. In this case, for example, end of the imaging operation atthis time, or continuation of the imaging operation while discardingimage data exceeding a half of the imaging range may be selected.

Operation Example of Imaging Apparatus

Next, the operation of the imaging apparatus 100 according to the firstembodiment of the present disclosure will be described with reference tothe drawings.

Operation Example in First Imaging Mode

FIG. 24 is a flowchart illustrating an example of the processingprocedure of imaging control processing by the imaging apparatus 100according to the first embodiment of the present disclosure. With thisexample, an example of the processing procedure in the event that thefirst imaging mode has been set will be shown.

First, the control unit 190 determines whether or not there has been adetermining operation of a subject to be situated in the referenceposition (step S901), and in the event that the determining operationthereof has not been performed, continuously performs monitoring. On theother hand, in the event that the determining operation thereof has beenperformed (step S901), the control unit 190 determines the startposition and end position of the panoramic image imaging operation basedon the position at the time of the determining operation thereof(reference position) (step S902). Note that step S901 is an example of adetermining procedure.

Next, the display control unit 191 controls the display unit 141 todisplay a guide screen (e.g., display screen 360 shown in FIG. 9B, swingoperation assistance screen 370 shown in FIG. 10B) up to the determinedstart position thereof under control of the control unit 190 (stepS903). Note that this guide screen will sequentially be changedaccording to change in the orientation of the imaging apparatus 100.

Next, the orientation detecting unit 121 detects change in theorientation of the imaging apparatus 100 (step S904), and the controlunit 190 determines whether or not the swing operation has correctlybeen performed based on the detected orientation change (step S905). Inthe event that the correct swing operation has not been performed (stepS905), the display control unit 191 controls the display unit 141 todisplay an error screen under control of the control unit 190 (stepS916), and ends the operation of the imaging control processing. On theother hand, in the event that the correct swing operation has beenperformed (step S905), the control unit 190 determined whether or notthe imaging apparatus 100 has arrived at the start position (step S906).In the event that the imaging apparatus 100 has not arrived at the startposition (step S906), the processing returns to step S903.

On the other hand, in the event that the imaging apparatus 100 hasarrived at the start position (step S906), the display control unit 191controls the display unit 141 to display a display screen indicatingthat the imaging apparatus 100 has arrived at the start position undercontrol of the control unit 190 (step S907). This display screen is, forexample, the swing operation assistance screen 380 shown in FIG. 11B.

Next, determination is made whether or not the start instructingoperation of the imaging operation has been performed (step S908), andin the event that the start instructing operation of the imagingoperation has not been performed, the display control unit 191continuously performs monitoring. On the other hand, in the event thatthe start instructing operation of the imaging operation has beenperformed (step S908), the display control unit 191 controls the displayunit 141 to display a guide screen up to the end position under controlof the control unit 190 (step S909). Note that this guide screen is, forexample, the swing operation assistance screen 381 shown in FIG. 12B,and will sequentially be changed according to change in the orientationof the imaging apparatus 100.

Next, the orientation detecting unit 121 detects change in theorientation of the imaging apparatus 100 (step S910), and the controlunit 190 determines whether or not the correct swing operation has beenperformed based on the detected orientation change thereof (step S911).In the event that the correct swing operation has not been performed(step S911), the processing proceeds to step S916. On the other hand, inthe event that the correct swing operation has been performed (stepS911), the imaging unit 112 performs image imaging processing forgenerating a panoramic image under control of the control unit 190 (stepS912). Note that steps S904 and S910 are an example of an orientationdetecting procedure. Also, step S912 is an example of a controlprocedure.

Next, the control unit 190 determines whether or not the imagingapparatus 100 has arrived at the end position (step S913). In the eventthat the imaging apparatus 100 has not arrived at the end position (stepS913), the processing returns to step S909.

On the other hand, in the event that the imaging apparatus 100 hasarrived at the end position (step S913), the display control unit 191controls the display unit 141 to display a display screen to the effectthat the imaging operation has been completed under control of thecontrol unit 190 (step S914). This display screen is, for example, theswing operation assistance screen 383 shown in FIG. 14B. Next, thepanoramic image generating unit 132 uses multiple images generated bythe imaging operation to generate a panoramic image, and the recordingcontrol unit 192 records the generated panoramic image in the recordingmedium 180 (step S915).

Operation Example of Second Imaging Mode

FIG. 25 is a flowchart illustrating an example of the processingprocedure of the imaging control processing by the imaging apparatus 100according to the first embodiment of the present disclosure. Thisexample illustrates an example of the processing procedure in the eventthat the second imaging mode has been set. Also, this processingprocedure is a modification of FIG. 24, and accordingly, portions commonto FIG. 24 are denoted with the same reference numerals, and descriptionthereof will be omitted.

In the event that the determining operation of a subject to be situatedin the reference position has been performed (step S901), the controlunit 190 determines the start position of the panoramic image imagingoperation based on the position at the time of the determining operationthereof (reference position) (step S921).

Also, in the event that the correct swing operation has been performed(step S905), the control unit 190 determines whether or not the startinstructing operation of the panoramic image imaging operation has beenperformed (step S922). In the event that the start instructing operationof the panoramic image imaging operation has not been performed (stepS922), the processing returns to step S903.

On the other hand, in the event that the start instructing operation ofthe panoramic image imaging operation has been performed (step S922),the control unit 190 determines the end position of the panoramic imageimaging operation based on the position at the time of the startinstructing operation thereof (step S923).

Operation Example in Third Imaging Mode

FIG. 26 is a flowchart illustrating an example of the processingprocedure of the imaging control processing by the imaging apparatus 100according to the first embodiment of the present disclosure. Thisexample illustrates an example of the processing procedure in the eventthat the third imaging mode has been set. Also, this processingprocedure is a modification of FIG. 24, and accordingly, portions commonto FIG. 24 are denoted with the same reference numerals, and descriptionthereof will be omitted.

First, the control unit 190 determines whether or not the startinstructing operation of the panoramic image imaging operation has beenperformed (step S931), and in the event that the start instructingoperation of the panoramic image imaging operation has not beenperformed, continuously performs monitoring. On the other hand, in theevent that the start instructing operation of the panoramic imageimaging operation has been performed (step S931), the control unit 190determines the end position of the panoramic image imaging operationbased on the position at the time of the start instructing operationthereof (step S932).

Also, the control unit 190 determines whether or not the determiningoperation of a subject to be situated in the reference position has beenperformed (step S933), and in the event that the determining operationthereof has not been performed, the processing proceeds to step S911. Onthe other hand, in the event that the determining operation thereof hasbeen performed (step S933), the control unit 190 newly determines theend position of the panoramic image imaging operation based on theposition at the time of the determining operation thereof (referenceposition) (step S934), and the processing proceeds to step S911.

2. Second Embodiment

With the first embodiment of the present disclosure, an example has beenshown wherein the imaging control is performed based on the determiningoperation of a subject to be situated in the reference position in apanoramic image by the user's operations. However, with regard to thelayout of subjects in the whole of a panoramic image, it may be assumedthat the user determine this while viewing a correlation between thewhole of the panoramic image and subjects. Therefore, with the secondembodiment of the present disclosure, an example will be shown wherein amarker representing a reference subject is situated in a compositionassistance image, and the panoramic image imaging operation iscontrolled so as to obtain the user's desirable composition. Note thatthe configuration of an imaging apparatus according to the secondembodiment of the present disclosure is generally the same as theexample shown in FIGS. 1 and 2. Accordingly, portions common to thefirst embodiment of the present disclosure are denoted with the samereference numerals, and a part of description thereof will be omitted.

The control unit 190 shown in FIG. 2 controls the units of the imagingapparatus 100 based on the operation signal from the operation receptionunit 151, and the orientation change information from the orientationdetecting unit 121. For example, in the event that a display instructingoperation for displaying a composition assistance image (e.g., acomposition assistance image display region 601 shown in FIG. 27B) hasbeen received, the control unit 190 controls the display unit 141 todisplay the composition assistance image. This composition assistanceimage is an image used for determining the composition of a panoramicimage. Also, this composition assistance image is displayed on thedisplay unit 141 in a state in which the movement direction (swingdirection) of the imaging apparatus 100 at the time of performing thepanoramic image imaging operation, and the longitudinal direction in thecomposition assistance image generally match. Also, the control unit 190situates a person-type icon (e.g., a person-type icon 602 shown in FIG.27B) in the start position in the movement direction of the longitudinaldirection of a composition assistance image, and controls the displayunit 141 to display this. This person-type icon is a reference subjectmarker representing a position in a panoramic image of a subject(reference subject) serving as an imaging object of the imaging unit 112at the time of a display instructing operation thereof being received.Also, the control unit 190 performs control for moving a person-typeicon in a composition assistance image from the start position toanother edge portion based on change in the orientation of the imagingapparatus 100 which heads for the opposite direction of the movementdirection (swing direction). Also, for example, a composition assistanceimage is a general rectangle, and the size in the longitudinal directionthereof matches the range of movement if the imaging apparatus 100 inthe direction of movement at the time of shooting a panoramic image.

Also, the control unit 190 appends a reference position markerrepresenting the reference position (e.g., the reference position mark604 shown in FIG. 27B) to the position of the composition assistanceimage corresponding to the reference position in the longitudinaldirection of a panoramic image, and displays the composition assistanceimage.

Also, in the event that the start instructing operation for instructingstart of the panoramic image imaging operation has been received, thecontrol unit 190 starts the panoramic image imaging operation with theposition where the start instructing operation thereof has been receivedbeing taken as the start position of the panoramic image imagingoperation. Also, the control unit 190 determines the end position of thepanoramic image imaging operation based on the start position thereof.Also, in the event that the start instructing operation thereof has beenreceived, the control unit 190 eliminates the composition assistanceimage, and displays an operation assistance image for changing theorientation of the imaging apparatus 100 up to the end position thereof.

The display control unit 191 controls the display unit 141 to displaythe images under control of the control unit 190.

Display Screen Example when Setting Panoramic Image Imaging Mode

FIGS. 27A and 27B are diagrams illustrating a display screen example tobe displayed on the input/output panel 200 according to the secondembodiment of the present disclosure.

FIG. 27A illustrates a display screen 315 to be displayed before thestart instructing operation of the panoramic image imaging operation.The display screen 315 is the same as the display screen 300 shown inFIG. 4, and “HALF-PRESS SHUTTER BUTTON WHILE FACING REFERENCE SUBJECT TOLOCK AF” is displayed in a message display region 316. Also, a subjectserved as an imaging object by the imaging unit 112 is displayed on thedisplay screen 315.

FIG. 27B illustrates a swing operation assistance screen 600. The swingoperation assistance screen 600 is a display screen to be displayed inthe event that the shutter button 210 has been half-pressed in a statein which the display screen 315 shown in FIG. 27A is displayed, and isdisplayed until start of the panoramic image imaging operation. Notethat, with the second embodiment of the present disclosure, an examplewill be shown wherein the display instructing operation of the swingoperation assistance screen 600 is performed by the half-pressingoperation of the shutter button 210, and the start instructing operationof the panoramic image imaging operation is performed by thefull-pressing operation of the shutter button 210.

Specifically, with the swing operation assistance screen 600, acomposition assistance image display region 601 and a message displayregion 605 are provided. Note that a focusing informing icon 375, ashutter speed informing icon 376, an F value informing icon 377, and aflash prohibition icon 378 are the same as those shown in FIG. 10B andothers, and accordingly denoted with the same reference numerals, anddescription thereof will be omitted here.

The composition assistance image display region 601 is a display regionin bar used at the time of determining the position of the referencesubject in a panoramic image. The composition assistance image displayregion 601 is displayed in a state in which the movement direction(swing direction) of the imaging apparatus 100 at the time of performingthe panoramic image imaging operation generally matches the longitudinaldirection of the composition assistance image. Here, the referencesubject is a subject displayed on the center of the display screen atthe time of the shutter button 210 being half-pressed in a state inwhich the display screen 315 is displayed, and an example of this is acattle image 401 shown in FIG. 28A.

Also, a person-type icon 602 virtually representing the position of apanoramic image situated where the reference subject determined by theuser's operations before start of the imaging operation is displayed inthe composition assistance image display region 601. The person-typeicon 602 moves in the composition assistance image display region 601according to change in the orientation of the imaging apparatus 100.

Specifically, in the event that the setting for performing the swingoperation from the left to the right has been set, when the shutterbutton 210 is half-pressed in a state in which the display screen 315 isdisplayed, the person-type icon 602 is situated in the left edge in thecomposition assistance image display region 601. The person-type icon602 then moves from the left to the right in the composition assistanceimage display region 601, according to change in the orientation of theimaging apparatus 100 (change heading from the right to the left). Also,a dotted-line arrow 603 representing this movement direction isdisplayed in the composition assistance image display region 601.

A reference position mark 604 representing the reference position in apanoramic image is displayed in the composition assistance image displayregion 601. With this example, an example is shown wherein a dotted lineis appended to the composition assistance image display region 601 asthe reference position mark 604. Note that the layout of the referenceposition mark 604 in the composition assistance image display region 601is the same as the reference position mark 379 according to the firstembodiment of the present disclosure (shown in FIG. 10B and others), andaccordingly, description thereof will be omitted here.

Now, a relative relation between the composition assistance imagedisplay region 601 and the person-type icon 602 will be described. Thecomposition assistance image display region 601 virtually represents theoutline of a panoramic image, and the person-type icon 602 virtuallyrepresents the position of the reference subject situated in thepanoramic image thereof. Specifically, the composition assistance imagedisplay region 601 corresponds to the panoramic image, and theperson-type icon 602 corresponds to a subject included in the panoramicimage.

Note that the movement of the person-type icon 602 in the compositionassistance image display region 601 (movement according to change in theorientation of the imaging apparatus 100) is the same as the operationposition 372 according to the first embodiment of the present disclosureexcept that the movement directions differ. Accordingly, descriptionthereof will be omitted here.

The message display region 605 is a region where a message for providingassistance for the user's operations relating to the panoramic imageimaging operation (the swing operation of the imaging apparatus 100) isdisplayed.

Imaging Control Example in Fourth Imaging Mode

FIGS. 28A and 28B are diagrams schematically illustrating an example ofthe orientation of the imaging apparatus 100 according to the secondembodiment of the present disclosure, and a display screen displayed onthe input/output panel 200. Also, the second embodiment of the presentdisclosure will show an imaging operation example in the event that thefourth imaging mode has been set, and a setting for performing the swingoperation from the left to the right as the swing direction has been setas the image size. Also, the second embodiment of the present disclosurewill show an imaging operation example in the event that the centerposition (e.g., the position of “0” in the reference position specifyingbar 351) is set as the reference position in a panoramic image.

FIG. 28A illustrates a case where the imaging apparatus 100, and theuser 415 holding the imaging apparatus 100 in both hands are viewed fromabove, in a simplified manner. In this case, let us say that the user415 confirms a reference subject (cattle image 401) in a panoramic imagewhile viewing a subject displayed on the input/output panel 200.

FIG. 28B illustrates a display screen 315 to be displayed before adetermination operation for determining the position of a referencesubject (reference position) in a panoramic image is performed. Thedisplay screen 315 is the same as the display screen 315 shown in FIG.27A. Also, a subject served as an imaging object by the imaging unit 112(cattle image 401) is displayed as a through image. Also, with regard toa case where the user 415 has performed a half-pressing operation of theshutter button 210 in the state shown in FIG. 28A, a swing operationassistance screen 600 shown in FIG. 27B is displayed. Also, with regardto a transition example of change in the orientation of the imagingapparatus 100, and the display screen after the swing operationassistance screen 600 is displayed, description will be made in detailwith reference to FIGS. 29A through 32C.

FIGS. 29A through 32C are diagrams schematically illustrating a relationexample of the orientation of the imaging apparatus 100 according to thesecond embodiment of the present disclosure, a display screen displayedon the input/output panel 200, and a reference subject in a panoramicimage. Specifically, FIGS. 29A, 30A, 31A, and 32A illustrate anorientation example of the imaging apparatus 100 at the time of afull-pressing operation of the shutter button 210. Note that, in FIGS.29A, 30A, 31A, and 32A, the user 415 and the imaging apparatus 100 areindicated with a dotted line. Also, FIGS. 29B, 30B, 31B, and 32Billustrate an example of the swing operation assistance screen 600displayed on the input/output panel 200 according to the orientation ofthe imaging apparatus 100 immediately before the full-pressing operationof the shutter button 210. Also, FIGS. 29C, 30C, 31C, and 32C illustratean example of a panoramic image generated at the time of thefull-pressing operation of the shutter button 210 being performed in thestates shown in FIGS. 29A, 30A, 31A, and 32A. Note that the examplesshown in FIGS. 29A through 32C are examples wherein a part of FIGS. 8Athrough 14B has been modified, and accordingly, portions common to FIGS.8A through 14B will be denoted with the same reference numerals, and apart of description thereof will be omitted.

FIGS. 29A through 29C illustrate a case where immediately after ahalf-pressing operation of the shutter button 210, a full-pressingoperation thereof is performed (or a case where a full-pressingoperation is performed without a half-pressing operation of the shutter210 being not performed).

For example, as shown in FIG. 28A, the user 415 holding the imagingapparatus 100 in both hands confirms the reference subject (cattle image401) in a panoramic image while viewing a subject displayed on theinput/output panel 200. In this case, as shown in FIG. 28B, the displayscreen 315 is displayed on the input/output panel 200.

FIG. 29A illustrates a case where the user 415 has performed afull-pressing operation of the shutter button 210 in the state shown inFIG. 28A, in a simplified manner. In this case, the position in theoptical-axis direction from a position 620 at the time of pressingthereof is determined to be a start position 621. An end position 622 ofthe imaging operation is then determined based on the start position621. Specifically, the end position 622 is determined from the startposition 621 of the imaging operation based on an image size θ21 (=180degrees) that has been set.

In this way, in the event that the user 415 has performed afull-pressing operation of the shutter button 210, the imaging operationis started from the position at the time of pressing thereof (startposition 621). Accordingly, in the event that the user 415 has performeda full-pressing operation of the shutter button 210 in the state shownin FIG. 29A, a panoramic image 611 (FIG. 29C) where the referencesubject (cattle image 401) is situated on the left edge is generated.

FIG. 29B illustrates the swing operation assistance screen 600 displayedon the input/output panel 200 immediately before the user 415 performs afull-pressing operation of the shutter button 210 in the state shown inFIG. 29A. In this case, the full-pressing operation is performedimmediately after a half-pressing operation of the shutter button 210,and accordingly, the person-type icon 602 is situated on the left edgeof the composition assistance image display region 601.

Also, the layout of the reference subject (cattle image 401) in thepanoramic image 611 shown in FIG. 29C generally matches the layout ofthe reference subject (cattle image 401) in the composition assistanceimage display region 601.

Note that FIGS. 29A through 29C illustrate an example in the event thatimmediately after a half-pressing operation of the shutter button 210, afull-pressing operation thereof is performed, but this case may also beapplied to a case where after the orientation of the imaging apparatus100 is changed, a full-pressing operation is performed. These exampleswill be shown in FIGS. 30A through 32C.

FIGS. 30A through 30C illustrate a case where after a half-pressingoperation of the shutter button 210, a full-pressing operation isperformed at timing wherein the change amount in the orientation of theimaging apparatus 100 is relatively small.

FIG. 30A illustrates a case where the user 415 has performedfull-pressing of the shutter button 210 at timing wherein the changeamount in the orientation of the imaging apparatus 100 is relativelysmall from the state shown in FIG. 28A, in a simplified manner. In thiscase, the position in the optical-axis direction from the position 620at the time of pressing thereof is determined to be the start position625. The end position 626 of the imaging operation is then determinedbased on the start position 625. Specifically, the end position 626 isdetermined from the start position 625 of the imaging operation based onan image size θ21 (=180 degrees) that has been set.

In this way, in the event that the user 415 has performed afull-pressing operation of the shutter button 210, the image operationis started from the position at the time of pressing thereof (startposition 625). Accordingly, in the event that the user 415 has performeda full-pressing operation of the shutter button 210 in the state shownin FIG. 30A, a panoramic image 612 (shown in FIG. 30C) in which thereference subject (cattle image 401) is situated in a position towardsthe left from the center position in the composition assistance imagedisplay region 601 is generated.

FIG. 30B illustrates the swing operation assistance screen 600 displayedon the input/output panel 200 immediately before the user 415 performs afull-pressing operation of the shutter button 210 in the state shown inFIG. 30A. In this case, a full-pressing operation is performed at timingwherein the change amount of the orientation of the imaging apparatus100 is relatively small after a half-pressing operation of the shutterbutton 210, and accordingly, the person-type icon 602 is situated in aposition towards the left from the center position in the compositionassistance image display region 601.

Also, the layout of the reference subject (cattle image 401) in thepanoramic image 612 shown in FIG. 30C generally matches the layout ofthe reference subject (cattle image 401) in the composition assistanceimage display region 601.

Also, FIGS. 31A through 32C illustrate a case where after ahalf-pressing operation of the shutter button 210, a full-pressingoperation is performed at timing wherein the change amount of theorientation of the imaging apparatus 100 is relatively great. Note thata determining method of start positions 631 and 635, and end positions632 and 636, and a method for situating the person-type icon 602 in thecomposition assistance image display region 601 are the same as those inFIGS. 29A through 30C, and accordingly, description thereof will beomitted here.

Also, the layout of the reference subject (cattle image 401) in thepanoramic image 613 shown in FIG. 31C generally matches the layout ofthe reference subject (cattle image 401) in the composition assistanceimage display region 601. Similarly, the layout of the reference subject(cattle image 401) in the panoramic image 614 shown in FIG. 32Cgenerally matches the layout of the reference subject (cattle image 401)in the composition assistance image display region 601.

Here, the example shown in FIGS. 31A through 31C illustrates, as shownin FIG. 31B, a case where a full-pressing operation is performed attiming when the person-type icon 602 being situated on the referenceposition mark 604 in the composition assistance image display region601. In this case, as shown in FIG. 31C, the reference subject (cattleimage 401) is situated in the center position (reference position) inthe panoramic image 613. Thus, a panoramic image where the referencesubject (cattle image 401) is situated in the center position (referenceposition) in the panoramic image 613 can be readily generated. Note thatan arrangement may be made wherein the person-type icon 602 is blinkedat timing when the person-type icon 602 is situated on the referenceposition mark 604, thereby informing the user of timing when thereference subject is situated in the reference position in the panoramicimage.

In this way, the user 415 may perform a full-pressing operation of theshutter button 210 at desired timing while confirming the position ofthe reference subject (cattle image 401) in the composition assistanceimage display region 601. Thus, the reference subject in the panoramicimage may be situated in a desired position, and accordingly, a user'sfavorite panoramic image may readily be generated.

Note that, with this example, a half-pressing operation of the shutterbutton 210 by the user 415 will be taken as a display condition for theswing operation assistance screen 600, and a full-pressing operation ofthe shutter button 210 will be taken as a determining condition for thestart position of the imaging operation, but another condition may beemployed. For example, a full-pressing operation for the shutter button210 may be employed as a display condition of the swing operationassistance screen 600, and a determining condition for the startposition of the imaging operation. Also, for example, an operation forpressing the shutter button 210 twice (what we might call clickingtwice), or a pressing operation of a dedicated button may be taken as adisplay condition for the swing operation assistance screen 600.

Also, imaging operation after start of the panoramic image imagingoperation is generally the same as the example shown in FIGS. 12Athrough 14B, and accordingly, description thereof will be omitted here.

Example Using Reference Subject as Icon

An example has been shown so far wherein the person-type icon 602 issituated in the composition assistance image display region 601, but theimage of the reference subject may be situated instead of theperson-type icon 602. Accordingly, with FIGS. 33A through 34B, anexample will be shown wherein the image of the reference subject issituated in the composition assistance image display region 601.

Specifically, the control unit 190 shown in FIG. 2 performs control forsituating the image of at least a part of a subject serving as animaging object of the imaging unit 112 at the time of receiving adisplay instructing operation for displaying a composition assistanceimage in the composition assistance image as a reference subject marker,and displaying this. Also, the control unit 190 performs control forsituating the image of at least a part of a subject serving as animaging object of the imaging unit 112 at the time of receiving thedisplay instructing operation thereof in the composition assistanceimage as a reference subject marker, and displaying this. In this case,the control unit 190 performs control so as to display the referencesubject marker in the composition assistance image (e.g., referencesubject image 651 shown in FIG. 34B) and the reference position marker(e.g., reference subject image 652 shown in FIG. 34B) as differentdisplay modes.

FIGS. 33A and 33B are diagrams schematically illustrating an example ofthe orientation of the imaging apparatus 100 according to the secondembodiment of the present disclosure, and a display screen displayed onthe input/output panel 200. Note that FIG. 33B is the same as FIG. 28Bexcept that an extraction frame 317 of the reference subject image isappended to the display screen 315 in FIG. 28B.

The extraction frame 317 is a frame used at the time of extracting theimage of the reference subject to be situated in the compositionassistance image display region 601. For example, let us assume a casewhere in the event of the display screen 315 shown in FIG. 33B beingdisplayed, a half-pressing operation of the shutter button 210 has beenperformed. In this case, the panoramic image generating unit 132extracts an image included in the extraction frame 317 (referencesubject image) from the image generated by the imaging unit 112. Thedisplay control unit 191 then controls the input/output panel 200 todisplay the swing operation assistance screen 600, situates theextracted image (reference subject image) in the composition assistanceimage display region 601 in the swing operation assistance screen 600,and displays this. In this case, according to the size of thecomposition assistance image display region 601, the extracted image(reference subject image) is reduced and displayed. A display example ofthis reference subject image is shown in FIGS. 34A and 34B. Note that amethod for situating and moving the reference subject image in thecomposition assistance image display region 601 is the same as with theperson-type icon 602, and accordingly, description thereof will beomitted here.

FIGS. 34A and 34B are diagrams illustrating a display screen example tobe displayed on the input/output panel 200 according to the secondembodiment of the present disclosure.

FIG. 34A illustrates an example wherein, with the swing operationassistance screen 600 shown in FIG. 27B, the reference subject image 651is situated and displayed instead of the person-type icon 602 in thecomposition assistance image display region 601. In this way, thereference subject image 651 is situated in the composition assistanceimage display region 601 and displayed, whereby the actual referencesubject (cattle image 401) on a panoramic image can readily be assumed,and determination of a composition can readily be performed.

FIG. 34B illustrates an example wherein, with the swing operationassistance image 600 shown in FIG. 34A, the reference subject image 652is situated and displayed instead of the reference position mark 604 inthe composition assistance image display region 601. Note that it isdesirable to display the reference subject image 652 and referencesubject image 651 as different display modes. For example, it isdesirable to display the reference subject image 652 so as to be lessrepresented as compared to the reference subject image 651. For example,a value a (numeric value representing transparency) is changed, wherebythe transparency of the reference subject image 652 can be displayed soas to be lower than the reference subject image 651. Note that FIG. 34Billustrates the reference embodiment of the present disclosure. Withthis example, an example of the processing procedure in the event thatthe fourth imaging mode has been set is shown. Also, this processingprocedure is a modification of FIG. 24, and accordingly, portions commonto FIG. 24 are denoted with the same reference numerals, and descriptionthereof will be omitted.

First, the control unit 190 determines whether or not a displayinstructing operation for a composition assistance image has beenperformed (step S951), and in the event that the display instructingoperation thereof has not been performed, continuously performsmonitoring. On the other hand, in the event that the display instructingoperation thereof has been performed (step S951), the control unit 190determines the start position and end position of the panoramic imageimaging operation based on the position at the time of the displayinstructing operation (step S952). Note that the start position and endposition are imaginary positions used for displaying a compositionassistance image.

Next, the display control unit 191 controls the display unit 141 todisplay the composition assistance image (e.g., swing operationassistance screen 600 shown in FIG. 27B (composition assistance imagedisplay region 601)) under control of the control unit 190 (step S953),for example.

Next, the orientation detecting unit 121 detects change in theorientation of the imaging apparatus 100 (step S954). Next, the controlunit 190 moves the reference subject marker in the compositionassistance image (e.g., the person-type icon 602 shown in FIG. 27B)based on the detected orientation change (step S955). Next, the controlunit 190 determines whether or not the correct swing operation has beenperformed based on the detected orientation change (step S956). In theevent that the correct swing operation has not been performed (stepS956), the processing proceeds to step S916.

Also, in the event that the correct swing operation has been performed(step S956), determination is made whether or not a start instructingoperation for the imaging operation has been performed (step S957), andin the event that a start instructing operation for the imagingoperation has not been performed, the processing returns to step S954.On the subject image 652 using a dotted line as an example to bedisplayed so as not to be inconspicuous as compared to the referencesubject image 651.

Thus, timing wherein the reference subject image 651 is overlapped withthe reference subject image 652 in the composition assistance imagedisplay region 601 becomes timing wherein a reference subject issituated in the reference position in a panoramic image. Accordingly, anoperation for situating a reference subject in a reference position in apanoramic image can be performed in a relatively easy-to-see manner.

Note that, with this example, an example is shown wherein an imageincluded in the extraction frame 317 having a particular size isextracted as a reference subject image, and this reference subject imageis situated in the composition assistance image display region 601.However, for example, an arrangement may be made wherein a facedetecting unit is provided to the imaging apparatus 100, a face portionalone of an image included in the extraction frame 317 having aparticular size is detected and extracted, and this face is situated inthe composition assistance image display region 601 as a referencesubject image. Note that in the event that there are many faces includedin the extraction frame 317 having a particular size, a face having ahigh evaluation value (e.g., smile face, great in size) may be employed.Alternatively, an arrangement may be made wherein a detecting unit fordetecting a particular object other than faces is provided to theimaging apparatus 100, and the particular object detected and extractedby the detecting unit thereof is situated in the composition assistanceimage display region 601 (particular object within the extraction frame317) as a reference subject image.

Operation Example of Imaging Apparatus

Next, the operation of the imaging apparatus 100 according to the secondembodiment of the present disclosure will be described with reference tothe drawings.

Operation Example in Fourth Imaging Mode

FIG. 35 is a flowchart illustrating an example of the processingprocedure of the imaging control processing by the imaging apparatus 100according to the second other hand, in the event that a startinstructing operation for the imaging operation has been performed (stepS957), the control unit 190 determines the end position of the panoramicimage imaging operation based on the position at the time of the startinstructing operation thereof (step S958).

Display Example when Vertical Direction is Taken as Swing Direction

With the first and second embodiments of the present disclosure, anexample has been shown wherein a panoramic image is generated byperforming the swing operation in the horizontal direction. However, asdescribed above, a panoramic image may be generated by performing theswing operation in the vertical direction. Therefore, FIGS. 36A and 36Billustrate a display screen example to be displayed on the input/outputpanel 200 in the event of performing the swing operation in the verticaldirection.

FIGS. 36A and 36B are diagrams illustrating a display screen example tobe displayed on the input/output panel 200 according to the first andsecond embodiments of the present disclosure.

FIG. 36A illustrates a swing operation assistance screen 700 to bedisplayed on the input/output panel 200 in the event that one of thefirst though third imaging modes has been set. With the swing operationassistance screen 700, a swing operation assistance image display region701, an arrow 703, and a message display region 704 are provided. Notethat the swing operation assistance screen 700 is the same as the swingoperation assistance screen 370 shown in FIG. 10B except that thedirections of the units, and so forth differ, and accordingly,description thereof will be omitted here.

FIG. 36B illustrates a swing operation assistance screen 710 to bedisplayed on the input/output panel 200 in the event the fourth imagingmode has been set. With the swing operation assistance screen 710, aswing operation assistance image display region 711, and a messagedisplay region 715 are provided. Note that the swing operationassistance screen 710 is the same as the swing operation assistancescreen 600 shown in FIG. 27B except that the directions of the units,and so forth differ, and accordingly, description thereof will beomitted here.

3. Modifications Modifications of Imaging Operation of Panoramic Image

With the first and second embodiments of the present disclosure, anexample has been shown wherein the panoramic image imaging operation isperformed by the user performing the swing operation of the imagingapparatus 100 with the position of the imaging apparatus 100 as areference. However, for example, the panoramic image imaging operationmay be assumed to be performed by moving the imaging apparatus 100 on astraight line generally in parallel with a subject. Therefore,description will be made below regarding an example wherein thepanoramic image imaging operation is performed by moving the imagingapparatus 100 on a straight line generally in parallel with a subject.

FIGS. 37A and 37B are diagrams schematically illustrating the imagingoperation at the time of generating a panoramic image by the imagingapparatus 100 according to the first and second embodiments of thepresent disclosure. FIG. 37A schematically illustrates a state viewedfrom above regarding the imaging operation for generating a panoramicimage by rotating the imaging apparatus 100 in the horizontal direction(swing direction 801) with the position of the imaging apparatus 100(the position of a user 800) as the rotation center. Also, FIG. 37Aschematically illustrates the imaging range of a panoramic imagegenerated by the imaging operation thereof using a thick line as apanoramic image imaging range 802. Note that the example shown in FIG.10A is the same as an example shown in the first and second embodimentsof the present disclosure, and accordingly, description thereof will beomitted here.

FIG. 37B schematically illustrates a state viewed from above regardingthe imaging operation for generating a panoramic image by moving theimaging apparatus 100 on a straight line 815 generally in parallel witha subject. The imaging operation shown in FIG. 37B is the panoramicimage imaging operation to be performed with a direction orthogonal toan advancing direction 811 as the optical-axis direction of the imagingapparatus 100 using a mobile device such as a vehicle or the like whichcan move at constant speed, for example. For example, let us say that avehicle in which a user 810 holding the imaging apparatus 100 in thehands rides moves at constant or variable speed in the advancingdirection 811. Also, FIG. 37B schematically illustrates the imagingrange of a panoramic image generated by the imaging operation thereofusing a thick line as the panoramic image imaging range 812.

In this way, with the example shown in FIG. 37B, the position of theimaging apparatus 100 is sequentially changed on the straight line 815,and accordingly, movement distance on the straight line is employedinstead of a rotation angle as change in the orientation of the imagingapparatus 100. Also, with regard to the start position and end positionof the panoramic image imaging operation as well, positions on thestraight line are employed.

Also, with the embodiments of the present disclosure, an example hasbeen shown wherein a panoramic image is generated by synthesizing stillimages, but the embodiments of the present disclosure may be applied toa case where a panoramic image is generated using a moving imagegenerated by a moving image imaging operation. Also, with regard to acase where a multi-view-point panoramic image is generated as well, theembodiments of the present disclosure may be applied.

Also, the embodiments of the present disclosure may be applied toimaging apparatuses such as cellular phones with an imaging function,portable terminal devices with an imaging function, and so forth.

Note that the embodiments of the present disclosure illustrate anexample for realizing the present disclosure, and elements according toembodiments of the present disclosure correspond to elements specific tothe present disclosure in the Claims correspond. Similarly, elementsspecific to the present disclosure in the Claims correspond to elementsaccording embodiments of the present disclosure denoted with the samenames. However, the present disclosure is not restricted to theseembodiments, and may be realized by subjecting the embodiments tovarious modifications without departing from the essence of the presentdisclosure.

Also, the processing procedures described in the embodiments of thepresent disclosure may be recognized as a method having a series ofprocedures, or may be recognized as a program causing a computer toexecute such series of procedures through a recording medium for storingthe program thereof. Examples of this recording medium include CD(Compact Disc), MD (MiniDisc), DVD (Digital Versatile Disk), memorycards, and Blu-ray Disc (registered trademark), and so forth.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. An image processing method for controlling animage capturing apparatus which captures images to generate a panoramicimage, the method comprising: determining an initial field of viewcorresponding to a center of the panoramic image based on a user input;determining, by processing circuitry of the image capturing apparatus,an imaging range based on the initial field of view of the imagecapturing apparatus, the imaging range including first and second fieldsof view on opposite sides of the center of the panoramic image;outputting a first visual indicator to a user to move the imagecapturing apparatus from the initial field of view toward the firstfield of view in a first direction to capture the first field of view;outputting a second visual indicator to the user to move the imagecapturing apparatus toward the second field of view in a seconddirection that is opposite to the first direction to capture the secondfield of view after the image capturing apparatus is determined to bemoved to capture the first field of view from the initial field of view;capturing the images corresponding to the first field of view and thesecond field of view based on the determined imaging range; andgenerating the panoramic image based on the captured images.
 2. Theimage processing method according to claim 1, wherein the panoramicimage is generated based on the captured images captured within theimaging range.
 3. The image processing method according to claim 1,wherein the imaging range is a range that includes an outer limit andanother outer limit.
 4. The image processing method according to claim1, wherein an outer limit of the imaging range is a position based on areference position.
 5. The image processing method according to claim 4,further comprising: determining the reference position based on amovement of the image capturing apparatus.
 6. The image processingmethod according to claim 4, further comprising: determining thereference position based on the user input.
 7. The image processingmethod according to claim 4, wherein the reference position isapproximately a center position between an outer limit of imaging rangeand another outer limit of imaging range.
 8. The image processing methodaccording to claim 1, further comprising: controlling a display of thecaptured images on a display of the image capturing apparatus.
 9. Theimage processing method according to claim 1, further comprising:obtaining information of a movement of the image capturing apparatuswithin the imaging range, wherein the capturing the images includescapturing the images corresponding to the the first field of view andthe second field of view based on the obtained information of themovement of the image capturing apparatus.
 10. The image processingmethod according to claim 9, wherein the obtained information of themovement represents at least one of a location of the image capturingapparatus and an inclination of the image capturing apparatus.
 11. Theimage processing method according to claim 9, further comprising:stopping the capturing of the images based on the obtained informationof the movement indicating that the image capturing apparatus hasreached an outer limit of the imaging range.
 12. The image processingmethod according to claim 9, further comprising: obtaining informationof the movement of the image capturing apparatus along the firstdirection, which is performed prior to starting of the capturing of theimages.
 13. The image processing method according to claim 12, furthercomprising: obtaining information of the movement of the image capturingapparatus along the second direction, which is performed after thestarting of the capturing of the images.
 14. The image processing methodaccording to claim 13, wherein the second direction is a direction thatextends from an outer limit of the imaging range toward another outerlimit of the imaging range.
 15. The image processing method according toclaim 9, further comprising: controlling a display of an indicationbased on the obtained information of the movement.
 16. The imageprocessing method according to claim 15, wherein the displayedindication is at least one of the first visual indicator representingthe first direction, the second visual indicator representing the seconddirection, and a third visual indicator representing a start of thecapturing.
 17. The image processing method according to claim 9, whereinthe information of the movement of the image capturing apparatus isobtained based on a detection by a sensor.
 18. A non-transitory computerreadable medium having a computer program recorded thereon, the computerprogram configured to perform a method when executed by an imagecapturing apparatus for controlling the image capturing apparatus whichcaptures images to generate a panoramic image, the method comprising:determining an initial field of view corresponding to a center of thepanoramic image based on a user input; determining an imaging rangebased on the initial field of view of the image capturing apparatus, theimaging range including first and second fields of view on oppositesides of the center of the panoramic image; outputting a first visualindicator to a user to move the image capturing apparatus from theinitial field of view toward the first field of view in a firstdirection to capture the first field of view; outputting a second visualindicator to the user to move the image capturing apparatus toward thesecond field of view in a second direction that is opposite to the firstdirection to capture the second field of view after the image capturingapparatus is determined to be moved to capture the first field of viewfrom the initial field of view; capturing the images corresponding tothe first field of view and the second field of view based on thedetermined imaging range; and generating the panoramic image based onthe captured images.
 19. The non-transitory computer readable mediumaccording to claim 18, the method further comprising: obtaininginformation of a movement of the image capturing apparatus within theimaging range, wherein the capturing the images includes capturing theimages corresponding to the first field of view and the second field ofview based on the obtained information of the movement of the imagecapturing apparatus.
 20. The non-transitory computer readable mediumaccording to claim 19, wherein the information of the movement of theimage capturing apparatus is obtained based on a detection by a sensor.21. An image capturing apparatus, comprising: an image sensor; andprocessing circuitry configured to determine an initial field of viewcorresponding to a center of a panoramic image to be generated based ona user input; determine an imaging range based on the initial field ofview of the image capturing apparatus, the imaging range including firstand second fields of view on opposite sides of the center of thepanoramic image; output a first visual indicator to a user to move theimage capturing apparatus from the initial field of view toward thefirst field of view in a first direction to capture the first field ofview; output a second visual indicator to the user to move the imagecapturing apparatus toward the second field of view in a seconddirection that is opposite to the first direction to capture the secondfield of view after the image capturing apparatus is determined to bemoved to capture the first field of view from the initial field of view;cause the image sensor to capture images corresponding to the firstfield of view and the second field of view based on the determinedimaging range; and generate the panoramic image based on the capturedimages.
 22. The image capturing apparatus according to claim 21, whereinthe processing circuitry is configured to obtain information of amovement of the image capturing apparatus within the imaging range, andcapture the images corresponding to the first field of view and thesecond field of view based on the obtained information of the movementof the image capturing apparatus.
 23. The image capturing apparatusaccording to claim 22, wherein the information of the movement of theimage capturing apparatus is obtained based on a detection by a sensor.24. The image capturing apparatus according to claim 21, wherein thefirst direction is a first horizontal direction and the second directionis a second horizontal direction.
 25. The image capturing apparatusaccording to claim 21, wherein the first and second fields of view areon opposites sides of the initial field of view.